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Chapter I 



Introduction 



I n 2006 and 2007, the U.S. Department of Education awarded Teacher Incentive Fund 
(TIF) grants for the development of innovative strategies for teacher compensation. 
New Leaders for New Schools (NLNS), with five partner organizations — Memphis City 
Schools (MCS), the District of Columbia Public Schools (DCPS), Denver Public 
Schools, Prince Georges County Public Schools, and a consortium of charter schools — 
received five of these grants and is using them to implement their Effective Practice 
Incentive Community (EPIC) intervention. EPIC offers performance-based awards to staff 
in high-performing schools in return for their agreement to work with NLNS in 
documenting and sharing effective teaching practices. 

Mathematica Policy Research Inc. (MPR) was hired to evaluate the EPIC initiative. Our 
work involves an analysis of the process for documenting and sharing effective teaching 
practices and an analysis of whether educators in participating districts change their practices 
after NLNS disseminates information on effective practices. When possible, MPR will also 
examine whether the availability of performance-based incentives leads to changes in student 
performance at schools eligible for those incentives. This report focuses on three EPIC 
partners of NLNS: the MCS and DCPS school districts and the consortium of charter 
schools. MPR is not conducting the evaluation of EPIC in Denver, and MPR’s work in 
Prince Georges County had not yet begun as of September 2008. 

EPIC awards were first given out in the fall of 2007 based on 2006-2007 performance. 
However, since school staff were unaware of the program before the fall of 2007 it is 
unlikely that the 2006-2007 performance was affected by EPIC eligibility. Thus, for the 
purposes of this report we are treating all data from before the 2007-2008 school year as 
baseline data. Due to data limitations we are presenting data from both the 2006-2007 and 
2005-2006 school years, depending on the variable and partner. Details regarding the year of 
the data are given in the partner chapters. 

Another component of the evaluation may involve comparing EPIC eligible and EPIC 
ineligible schools. In Memphis we have data on a number of EPIC ineligible schools. A 
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second purpose of this report will be to compare the baseline characteristics of the EPIC 
eligible and EPIC ineligible schools to see if there are any important differences that need to 
be taken into account when developing our evaluation plans. 

This chapter provides a general overview of EPIC, compares student characteristics 
across the three partners, and compares the data used to determine which schools receive 
EPIC awards with data on all the students attending these schools. Chapters II, III, and IV 
present baseline characteristics by partner: Chapter II focuses on Memphis, Chapter III on 
DC, and Chapter IV on the charter school consortium. The results are broken out by 
whether or not the schools received awards, by their performance levels, and by whether or 
not they were eligible for EPIC. 

Two additional reports on the characteristics of EPIC schools are forthcoming. In 
January 2009, MPR will release a summary of responses to our survey of principals in the 
MCS and DCPS districts and the charter school consortium. That summary will examine 
principals’ knowledge and perceptions of EPIC at baseline and their perceptions of incentive 
programs in general, and will report on the baseline practices followed by principals and 
teachers in EPIC-eligible (and in some cases, EPIC-ineligible) schools. In July 2009, MPR 
will release a report summarizing teacher awareness of EPIC; that report will also present 
case studies of the NLNS process for identifying effective practices in each of the three 
school districts. 

Overview of Epic 

EPIC has two key features. First, through the initiative NLNS offers performance- 
based incentives to eligible schools in each of the partner school districts. The awards are 
made to principals, teachers, and (in some cases) additional staff. NLNS identifies high- 
performing schools based on scores derived from Value Added Models (VAMs); VAM- 
based scores are similar to average student achievement gains. The awards are designed 
primarily to encourage school staff to help NLNS document effective practices. In addition, 
because these performance-based awards will be offered over multiple years, they might 
serve as an incentive for principals and teachers to improve student achievement. 

Second, EPIC includes an intensive effort to document and disseminate effective 
practices. NLNS works with staff in the highest-performing schools to identify practices that 
could contribute to the observed large growth in student achievement. These practices are 
then disseminated not only to all schools in the partnerships but to all schools nationwide, 
even those not eligible for incentive awards. The primary motivation for EPIC is the belief 
that these dissemination activities will result in changes in principal and teacher practices, 
and thereby cause improvements in student achievement. 



1 VAM scores are intended to reflect the average contribution that each school makes to student 
achievement, holding constant factors that the school cannot control, such as the prior achievement of each 
student. See Booker and Isenberg (2008) and Booker, Chaplin, and Isenberg (2008) for details. 
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Incentive Awards 

NLNS is making financial awards to staff in high-achieving schools during the 
2007/2008 through 2011/2012 school years. The payments are supplemental income that 
the staff receiving them may use as they see fit. 

In the 2007/2008 school year, NLNS made two types of EPIC awards: 

1. The Gold-Gain Award is given to the highest-performing schools in each 
partner district/ consortium. 

2. The Silver-Gain Award is given to the second tier of high-performing schools 
in each partner district/ consortium. 

The eligibility criteria for these EPIC incentive awards varied from partner to partner. 
In Memphis, only schools where 50 percent or more of the student population qualified for 
a free or reduce price lunch (F/RPL) were eligible, while in the District of Columbia all 
schools were eligible. Concerning the charter school consortium: only schools where 30 
percent or more of the student population qualifies for F/RPL were recruited into the 
consortium and could therefore become eligible for the awards. 2 As discussed later, other 
factors could affect a school’s eligibility for EPIC incentive awards. 

The achievement criteria for these two awards also varied among partners. In Memphis 
and the charter consortium, Gold-Gain awards went to schools with the highest VAM 
scores, and Silver-Gain awards to schools in the next tier of VAM scores. In the District of 
Columbia only Gold-Gain schools were identified, and this was done by examining changes 
in the percentage of students at each school that were proficient on state tests. 

The number of schools that received Gold-Gain and Silver-Gain awards varied from 
partner to partner. The number of awardees was determined by NLNS based in part on the 
distribution of high-performing schools. Other district-specific criteria were used to 
determine the number of schools in each category. 

The types of staff who received awards, as well as the size of the awards, also varied 
from partner to partner. Principals, assistant principals, and instructional staff received 
awards in Gold-Gain and Silver-Gain schools in all three partners. In DCPS, other building 
staff (guidance counselors and school support such as custodial and lunch service staff) also 
received awards. The size of the award payments varied substantially. For example, 
principals at Gold-Gain schools received between $10,000 and $20,000, and all instructional 
staff in Gold-Gain schools received between $1,500 and $8,000 (Table 1.1). 

A central goal of the 2007/2008 awards was to make teachers aware of the EPIC 
incentive program. While these early awards were given out based on performance during 
the 2006/2007 school year, the school staff were not aware of EPIC during 2006/2007. 



2 Eligibility was also conditional on supplying sufficient data for calculation of the VAM scores. 
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Awards to be made in 2008/2009, by contrast, will be based on performance during the 
2007/2008 school year, when staff were aware of the program, so knowledge about eligibility 
for these awards (and the awards given in later years) might encourage school staff to raise 
student achievement. 

In addition to the school-wide awards made in Gold-Gain and Silver-Gain schools, 
NLNS is planning to give “Spotlight” teacher awards to selected teachers in the Gold-Gain 
charter schools during the 2008/2009 school year and in Memphis during the 2009/2010 
school year. NLNS is not planning to make spotlight teacher awards in DCPS. Spotlight 
teachers will be identified through a combination of teacher-level VAM estimates, classroom 
observations, and interviews with building staff. 



Table 1.1. EPIC Incentive Award Amounts 





Gold-Gain Schools 


Silver-Gain 

Schools 


Memphis 


Principals 


$15,000 


$10,000 


Assistant principals 


10,000 


7,500 


All instructional staff 


1,500 


1,000 


Spotlight teachers (2009/2010 and later) 3 


7,500 


n.a. 


DCPS b 


Principals 


10,000 


n.a. 


Assistant principals 


9,000 


n.a. 


All instructional staff 


8,000 


n.a. 


Spotlight teachers 0 


n.a. 


n.a. 


Guidance counselors 


4,000 


n.a. 


School support 


2,000 


n.a. 


Charter School Consortium 


Principals 


20,000 


15,000 


Assistant principals 


15,000 


10,000 


All instructional staff 


1,500 


750 


“Spotlight” teachers (2008/2009 and later) 3 


10,000 


n.a. 



a Spotlight teacher awards are made to selected teachers in Gold-Gain schools only. 
b DCPS’s EPIC program — called TEAM — does not include payments to Silver-Gain schools. 
c DCPS has not decided whether or when it will make “Spotlight” teacher awards. 



Effective Practices 

The second key feature of EPIC is the documentation and dissemination of effective 
practices. Once the Gold-Gain schools have been identified, NLNS will work to identify the 
most effective teachers in those schools. NLNS will send what it calls “Effective Practice 
Teams” into Gold-Gain schools to identify effective principal and teacher practices. 

The Effective Practice Teams will document effective practices through a series of 
systematic interviews. The identification of these practices will also be informed by previous 
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research and through classroom observations. The Effective Practice Teams will film some 
of the effective practices in action for future dissemination. 

Once the effective practices are identified and documented, NLNS will develop a 
dissemination campaign. In each partner district or consortium, all schools - regardless of 
eligibility for incentive awards - will be targeted by the dissemination. Current plans call for 
dissemination through internet-based presentations and videos but NLNS is also considering 
other modes. 

Summary Research Approach 

The evaluation will focus on both the incentive award component and the effective 
practices component of EPIC. Key research questions include (but are not limited to): 

• In what ways do teachers and school administrators learn of the availability of 
EPIC incentives? 

• What are principals’ and teachers’ perceptions of performance-based pay? How 
do these perceptions change over time? How do these perceptions differ by 
partner? 

• What are principals’ expectations regarding whether incentives will affect 
teacher behavior and student outcomes? 

• How do the availability of awards and the dissemination of best practices affect 
student achievement in eligible schools? 

• Before EPIC’s best practices are disseminated, how do principals and teachers 
learn about best practices for teaching in their grade/ subject? Do these avenues 
of best-practice awareness differ by partner? 

• How do EPIC Effective Practice Teams identify best practices? How are they 
disseminated? 

• Are principals and teachers aware of EPIC -identified effective practices after 
they are disseminated? Does this differ by partner? 

• In what ways do principals and teachers report changing their teaching practices 
in response to EPIC effective-practice dissemination? Do the changes vary by 
partner? 

It is important to note that not all of these research questions will be examined for each 
partner. In particular, when examining the impact of incentives, we would like to know how 
students would have performed if their teachers were not eligible for an incentive award. 
However, for at least one — and possibly all — of the three EPIC partners, we will not have 
an appropriate comparison group that could be used to answer this question. For instance: 



I: Introduction 




6 



1. For DCPS, we have no valid comparison group to represent ineligible schools, 
because virtually all DCPS schools were eligible for EPIC. 

2. For MCS, EPIC eligibility criteria established at the outset that some schools 
could receive EPIC incentives and some schools were ineligible. These ineligible 
schools could be used as a comparison group to measure the impact of incentive 
eligibility on student achievement. However, as discussed later in this report, 
these ineligible schools are systematically different from the eligible schools in 
their baseline characteristics, casting doubt over the assumption that the 
ineligible schools would be a valid counterfactual for the eligible schools. 
Moreover, questions about the longevity of EPIC incentive awards in MCS 
could undermine the ability of the incentives to have an impact, thereby 
reducing the benefits of conducting an evaluation there. 

3. For the charter school consortium, there is a special challenge: finding a group 
of charter schools that has been participating in the intervention long enough to 
be affected and for which there is a reasonable comparison group. Schools that 
participated last year but not this year will not work, since their award last year 
was based on performance in the previous year when they were unaware of 
EPIC. Schools that participate this year for the first time are in a similar 
position — their awards this year will be based on last year’s performance, when 
they were not aware of EPIC. Thus, the most likely candidates for our treatment 
group will be schools that were participating in EPIC both this year and last 
year. Within that subset of participating charter schools we will then need to 
find reasonable comparison schools. We believe that it will probably be 
important to identify charter schools located in the same cities in order to 
accomplish this. We have identified several cities where we expect to have at 
least four charter schools each in the consortium. We hope to use other charter 
schools that are in these cities, but not in the consortium, as a comparison 
group for the evaluation. However, it remains to be seen how many of the 
charter schools participating in EPIC in the first year in those cities will 
continue participating in EPIC this year. To the extent that a high proportion of 
schools in these cities drops out of the consortium, our ability to evaluate the 
impact of incentives on student achievement will be compromised. 

In short, for partners for which we have no comparison group, our research will likely 
focus on the degree to which principals and teachers are aware of their eligibility and their 
perceptions of EPIC and teacher incentives more generally, but not on whether eligibility for 
EPIC affects student performance. In cases where there is a comparison group, we will 
examine how student performance is affected by EPIC eligibility. 

Overview of Student Data 

There are two sources of data used in this report: data on individual students in tested 
grades obtained in order to estimate value-added models (the VAM data) and data 
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summarizing school, student, and teacher characteristics that was obtained from various 
websites (public data). 

The VAM Data 

The VAM data for Memphis and the District of Columbia came directly from the 
school districts, which passed them on to NLNS, and from there the data were sent to MPR. 
Consequently, the way the data were provided was consistent across all schools within each 
of these districts. In contrast, the charter school VAM data came from each school 
separately, so MPR needed to do a great deal of work in order to put those data into one 
consistent format. The VAM performance measures are designed to measure performance 
during the 2006/2007 school yeari and all VAM student characteristics are for the 
2006/2007 school year. 

The charter school data were originally submitted in a variety of formats including flat 
ASCII files, SPSS, Microsoft Word, and Microsoft Excel. After all files were transferred into 
Excel, the data sets were reviewed for accuracy and completeness. All variables were coded 
consistently across files, and schools with missing data were contacted to request follow up 
data submissions. Several schools also chose to submit entry and exit dates for students in 
lieu of number of days enrolled. MPR staff used these dates to generate an approximation of 
days enrolled, accounting for weekends and major holidays. After two months of data 
cleaning, the data sets were stacked to create a "master" charter data set. Student IDs were 
then linked across years to create the longitudinal data set used to estimate the value-added 
model. 

For each partner, student data are being collected for 2005/2006 through 2010/2011. 
These data are used to construct VAM analyses of performance — and to identify Gold-Gain 
and Silver-Gain award schools — in the 2007/2008 through 2011/2012 school years. Note 
that the information available on student records differs from partner to partner (Table 1.2). 

The Public Data 

In this report, we rely primarily on the VAM data to describe the baseline characteristics 
of EPIC schools. The VAM data were selected because they reflect the characteristics of 
students who are being used to estimate the VAM scores and in the evaluation of EPIC. 
However, the VAM data do not cover all students in a school. Instead, they only include 
students who took the standardized tests included in the VAM estimates. This means that 
students in some nontested grades as well as students who were exempt from the tests are 
not included in the administrative data. We use publicly available data on all students in each 
school to see if there are differences between those students who are covered by VAM and 
those who are left out. We also use publicly available data to obtain information on some 
school characteristics that are not available in the VAM data. 



3 See Booker and Isenberg (2008), and Booker, Chaplin, and Isenberg (2008) for details. VAM scores in 
DC were estimated using a method similar to that used for Memphis. 



I: Introduction 




Table 1.2. Student Information Available by Partner 





Memphis 


DCPS 


Charter School 
Consortium 


Grade 




■/ 


S 


Academic performance 




■/ 




Days enrolled 


/ 




■/ 


Days present 


/ 






Race/ethnicity 


/ 




■/ 


Gender 


/ 


■/ 


■/ 


Eligible for free/reduced-price lunch 


/ 


■/ 


■/ 


Limited English proficiency status 




■/ 


■/ 


Special education status 




■/ 


■/ 


Suspended days 








Student retained 




■/ 




Expulsion status 








Excused and unexcused absences 




■/ 





In Memphis, the public data were obtained from the National Center for Education 
Statistics (NCES) Common Core of Data and “School Profiles” on the Memphis City 
Schools website. In DCPS, the public data were obtained from the NCES Common Core of 
Data and Annual Yearly Progress (AYP) reports linked to the DCPS website. 4 For charter 
schools, the public data came from a variety of sources. Most charter school data came from 
the NCES Common Core of Data and individual school report cards posted on state 
department of education websites. 3 Some data also came from individual school websites 
and the Center for Education Reform website. 

Differences Between the VAM Data and the Public Data 

Because the characteristics of students included in the VAM data could differ from the 
characteristics in the public data, we compared the VAM with public data on the fraction of 
students who are black, the fraction F/RPL, the fraction Hispanic, and the fraction male for 
all partners. For Memphis and the charter consortium we also compared the fractions 
classified as special ed and LEP. We found only one area where the difference was larger 
than four percentage points: the fraction of students LEP was 8 percent for the charter 
consortium in the VAM data but 16 percent in the public data. The remaining differences 
were generally quite small. (See Tables 1.3 — 1.5 for details.) 



4 AYP is used to measure school performance for the purpose of school accountability under the No 
Child Left Behind legislation (NCLB). 

5 States are required to post these report cards because of the NCLB legislation. 
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Table 1.3. Student Sample vs. Overall School Characteristics, Memphis City Schools 



Characteristics 


Tested Students (%) 


All Students (%) 


Ethnicity 

Black 


92.3 


90.7“* 


Hispanic 


3.8 


4 4 *** 


FRPL 


85.9 


85.8 


Male 


50.6 


51.6“* 


Number of Schools 


144 





‘Significantly different from the tested students at the 10% level 
“Significantly different from the tested students at the 5% level. 
“‘Significantly different from the tested students at the 1% level. 



Note: The data for tested students are for the students in our VAMs for the 2006-2007 school 

year. The data for all students are from the Common Core of Data for 2005-2006 
school year. 



Table 1.4. Student Sample vs. Overall School Characteristics, Charter Schools 



Characteristics 


Tested Students 
(%) 


All Students 
(%) 


Number of Schools with 
Nonmissing Values for Both 
Variables 


Ethnicity 

Black 


45.8 


46.1 


63 


Hispanic 


30.8 


30.6 


63 


FRPL 


68.2 


67.7 


54 


LEP 


8.3 


16.4*“ 


54 


Special education 


9.8 


9.1 


35 


Male 


50.8 


48.3 


24 



‘Significantly different from the tested students at the 10% level. 

“Significantly different from the tested students at the 5% level. 

‘“Significantly different from the tested students at the 1% level. 

Note: The data for tested students are for the students in our VAMs for the 2006 2007 school 

year. The data for all students are from publicly available data for 2006-2007 when 
possible and from 2005-2006 otherwise. 
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Table 1.5. Student Sample vs. Overall School Characteristics, DC Schools 



Characteristics 


Tested Students (%) 


All Students (%) 


Ethnicity 

Black 


83.8 


82.0“* 


Hispanic 


10.1 


10.9“ 


FRPL 


66.2 


67.8* 


LEP 


6.4 


8.5*“ 


Special Education 


15.6 


11.9*** 


Male 


51.8 


48.8*** 


Number of Schools 


126 





‘Significantly different from the tested students at the 10% level. 

“Significantly different from the tested students at the 5% level. 

“‘Significantly different from the tested students at the 1% level. 

Note: The data for tested students are for the students in our VAMs for the 2006-2007 school 

year. The data for all students are from the Common Core of Data for 2005-2006 
school year. 



There are a number of reasons why the VAM and public data could differ. One reason 
is that the data were sometimes from different years. 6 Another reason is that not all students 
were tested. While the differences observed were usually small, one difference was large and 
many of the others were statistically significant. Consequently, we separated out those 
schools where all students were in tested grades (that is, where no students were outside of 
the grade 3-8 range). Tables A.1-A.3 in the appendix show the data for schools that include 
tested grades only. 

In Memphis, three of the four differences between the VAM data and the public data 
are statistically significant when we look at all schools. One of these differences becomes 
statistically insignificant (for the fraction Hispanic) and the p-value for another increases 
from less than 0.01 to 0.07. Again, this is in part because the sample size was reduced to only 
19 schools out of an original 144. However, the magnitudes of both of those differences 
decrease as well. One difference remains statistically significant and similar in magnitude (for 
the fraction of students male), but still under 2 percentage points. 

Only one difference is statistically significant when we look at all charter schools, 
namely, the fraction of students LEP. This difference becomes smaller in magnitude and 
statistically insignificant when we limit the data to the five schools that only served tested 



6 The public data on student characteristics are for the 2005/2006 school year for DC and Memphis. In 
Charters the public data are for 2006/2007 when possible but for 2005/2006 when it was difficult to obtain 
2006/2007 data. 



I: Introduction 




11 



grades. However, one other difference goes from being under 1 percentage point and 
statistically insignificant to being over 4 percentage points and statistically significant, 
namely, the fraction of students in special education. This fraction was 13.8 percent in the 
VAM sample but only 9.5 percent in the public data, suggesting that in these five schools the 
Special Education students were overrepresented rather than being under-represented in the 
sample of students tested; it is the opposite of the pattern one would expect to see if schools 
were trying to avoid testing their low-performing students. 

In DCPS, all of the differences are statistically significant when we compare the VAM 
and public data using all schools, but none of the differences are statistically significant when 
we limit the data to the schools where all students were tested. This is in part because only 
12 schools tested all students. However, the magnitudes of all of those differences are also 
smaller in the latter set of schools. 

To summarize, it appears that the VAM data and public data generally yield similar 
pictures of school characteristics. There may be differences for individual schools but those 
differences offset each other when we analyze schools as a group, which is what will be done 
in our evaluation. 

Overview of Baseline Characteristics by Partner 

In this section we describe the characteristics of schools for each of the three partners. 
We cover the grade level, whether an award was received, the AYP status, the fractions of 
students who were proficient, the numbers of students and teachers, and the fractions of 
students by race, ethnicity, and gender. These statistics are presented by partner in Table 1.6. 
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Table 1.6. Baseline Characteristics 



Category 


Memphis Schools 


DC Schools 


Charters 


School Level (%) 


Elementary schools 


69.7 


72.9" 


34.0*** 


Middle schools 


16.6 


14.0 


47.4*** 


High schools 


13.8 


13.2 


18.6 


Awards (%) 


Gold-winning schools 


2.8 


2.3 


7.2* 


Silver-winning schools 


8.3 


0.0" 


15.5*** 


Non-award schools 


89.0 


97.7" 


77.3*** 


School Performance (%) 


AYP pass 


80.1 


24.6"* 


54.9*** 


Identified as “in need of improvement” 


16.4 


76.9"* 


36.3" 


Student proficiency 


ELA proficient 


83.6 


38.7*" 


40.0*** 


Math proficient 


80.3 


31.2"* 


39.7*** 


School Characteristics 


Enrollment 


641.6 


379.9*** 


421.1*** 


Student-teacher ratio 


16.9 


13.7*** 


17.1 


Staff size 


37.0 


31.0*** 


27.3*** 


Student Characteristics (%) 
Ethnicity 


Black 


91.9 


83.8" 


50.2*** 


Hispanic 


3.9 


10.1*** 


25.3*** 


FRPL 


85.9 


66.2*** 


65.9*** 


LEP 


3.1 


6.4*** 


7.0*** 


Special education 


10.9 


15.6*** 


10.6 


Number of Schools 


145 


129 


97 



'Significantly different from the Memphis City Schools at the 10% level 
"Significantly different from the Memphis City Schools at the 5% level. 
"'Significantly different from the Memphis City Schools at the 1% level. 



Notes: Only Memphis schools eligible for an EPIC award in 2008-2009 are included in these statistics. 
2006 school years. See text for details. 



These data are for the 2006-2007 and 2005- 
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For this report schools are divided into three grade-level categories (elementary, middle, 
and high) based in large part on how the EPIC awards were given out. The fractions of 
schools in each category are similar in DCPS and Memphis but quite different among the 
charter schools. This may have occurred in part because of the definitional differences across 
the partners 7 and in part because charter schools chose to apply to EPIC whereas in DCPS 
and Memphis most non-charter regular public schools were eligible for the program. 8 In DC 
and Memphis, about 70 percent of schools were categorized as elementary and less than 17 
percent were categorized as middle schools. In contrast, in the charter consortium only 
about one-third of schools were in the elementary school category and almost half were in 
the middle school category. 

NLNS worked with each partner and the U.S. Department of Education to determine 
the number of awards that would be given out. Factors that were considered included the 
total funding available for each partner and the distribution of VAM scores and their 
confidence regions, but there was no attempt to make the criteria for getting an award 
similar across partners. A much higher fraction of charter schools received awards than of 
DCPS or Memphis schools. DCPS only gave out one type of award and only about 2.3 
percent of schools received this award in the 2007-2008 school year (three schools in total). 
In Memphis, a similar percent (2.8) received Gold awards and another 6.2 percent received 
Silver awards. Among the charter schools, 7.2 percent of schools received Gold awards and 
15.5 percent received Silver awards. 

Each state determines its own cut-points for student proficiency. Thus, comparisons 
across states might not reflect differences in student skills (NCES 2007). However, 
comparisons do affect which schools may be held accountable under NCLB. In our data, 
Memphis schools had much higher student proficiency rates on state tests than did DCPS 
schools, perhaps in part because the State of Tennessee may have somewhat lower standards 
for proficiency than other states (NCES 2007). Over 80 percent of students in Memphis 
were proficient in both math and English Language Arts (ELA), in comparison with less 
than 40 percent of students in DCPS. The results for charters are omitted since the fractions 
of students proficient were available for only 13 out of the 97 charter schools. 



7 In DCPS, elementary schools are defined as schools that serve at least grades 4 and below, middle 
schools are schools that serve no grades lower than 5th or higher than 9th (middle schools include junior highs 
that go up through 9th grade), and high schools are defined as those that serve no grades lower than 9th. In 
Memphis, elementary schools typically serve grades 5 and below; a few include grade 6 as well. Most middle 
schools cover grades 6, 7, and 8. Most schools in the high school group have only students in 9th grade and 
above, but three high schools also include grades 7 and 8. Charter elementary schools serve grades 2 through 6 
(or any subset of those grades), charter middle schools serve grades 7 and 8, and charter high schools serve 9th 
grade and above. Unlike Memphis and DCPS, charter schools serving multiple school levels were eligible for 
multiple award pools, although they could only win one award apiece. In this report, most charter schools that 
covered multiple grade levels were assigned to their highest school level served. The exception is for schools 
that won an award at a level other than their highest level and that were assigned to the award-winning school 
level. 



8 The details of eligibility differed for Memphis and DCPS. See Chapters II and IV. 
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The Memphis schools were the largest schools based on both students and teachers. 
DCPS schools had fewer students, on average, but more teachers. Consequently, the ratio of 
students to teachers was lowest in DCPS schools (at around 14 to 1) compared with around 
17 to 1 for both Memphis and the charter schools. 

The student populations in both Memphis and DCPS schools were predominantly black 
(92 percent and 84 percent, respectively). In contrast, the charter schools averaged around 50 
percent black, although another 25 percent of their students were Hispanic, as compared 
with 10 percent Hispanic in DCPS and only about 4 percent in Memphis. 

In Memphis schools eligible for EPIC, on average about 86 percent of students were 
eligible for free or reduced price lunches (F/RPL), compared with about 66 percent of 
students in DCPS and charter schools.' 1 This is probably in large part because Memphis had 
a higher cut-off to be eligible for EPIC. In Memphis, schools had to have at least half of 
their students F/RPL to be eligible for EPIC, compared only a 30 percent for charters and 
no cut-off at all for DCPS. 10 

The fractions of students with limited English proficiency (LEP) were generally quite 
low. In Memphis, the average school had only 3.1 percent of students LEP. The fractions 
were larger for DCPS and the charter schools, averaging 6.4 and 7.0 percent respectively. 
The fractions of students designated for Special Education were also fairly low — averaging 
around 11 percent for Memphis and the charter schools and around 16 percent for DCPS. 

To summarize, in DCPS and Memphis most schools were elementary schools, whereas 
among the charters only about a third fell into that grade span. The fraction of schools 
receiving awards was highest for charters and lowest for DCPS, while the fractions of 
students measured as proficient was highest in Memphis. However, it should be noted that 
neither of these measures of performance was designed to be compared across these 
geographic entities. Memphis and DCPS had somewhat higher fractions of black students 
whereas charter schools had more Hispanics. Memphis’s eligible schools had a higher 
fraction of students eligible for F/RPL. The fractions of students LEP and in Special 
Education were low for all three sets of schools. 

Plan of the Report 

This chapter provided overviews of EPIC in general, the data used to examine the 
baseline characteristics of schools in the EPIC program, and overall characteristics of the 
EPIC schools. Subsequent chapters discuss the baseline characteristics of schools in 
Memphis, DCPS, and the charter school consortium, respectively. Because the specific 



9 The federal TIF funds can only be used to pay for awards to schools with at least 30% F/RPL. If 
schools with lower percentages of students F/RPL win awards DCPS pays for those awards with separate 
funding. 

10 In DCPS, funds from DoE could only be used for schools with at least 30 % F/RPL. If other schools 
had been given awards, DCPS would have needed to secure separate funding for those awards. 
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details of the EPIC initiative differ from partner district to partner district, each chapter 
summarizes how EPIC was implemented in each district. 

Each of the following chapters presents the baseline characteristics of schools, drawing 
comparisons between different types of schools within the partnership covered in that 
chapter. For each EPIC partner, different sets of comparisons are relevant: 

• For Memphis, where some schools are ineligible for EPIC, we compare the 
characteristics of eligible versus ineligible schools. This information is used to 
help inform our proposed efforts to estimate the impacts of EPIC in Memphis. 

• For all three partners, we compare EPIC award-winning schools with non- 
award-winning schools. 

• For all three partners, in order to understand how the highest achieving schools 
in each district (including but not limited to EPIC award winners) compare with 
schools at other points in the distribution of student achievement, we compare 
school characteristics by VAM quartile. Specifically, we compare the 
characteristics of schools in the top quartile of VAM estimates with schools in 
the middle two quartiles and bottom quartiles. Note that award winners are 
determined by VAM performance for Memphis and the charters, so the top 
quartile of the VAM performers overlaps substantially with the award winners. 

This is less true in DCPS, since EPIC awards there were not determined by 
VAM scores. 

• For the charter school consortium, because the schools are located in multiple 
states, we compare schools by their location. 

The findings from this analysis lead to the following conclusions about the baseline 
characteristics of schools in EPIC partner districts: 

• The schools that won EPIC awards resemble non-award-winning schools in 
terms of many student characteristics. However, schools that won EPIC awards 
are more likely to score high on other measures of achievement than schools 
that did not win EPIC awards. 

• When grouped by VAM quartile, schools appear similar in terms of school-level 
characteristics (enrollment, student teacher ratios, and so on). 

• When grouped by VAM quartile, schools in MCS appear similar in terms of 
student characteristics (race, LEP, percent F/RPL, and so on); schools in the 
charter consortium differ on some characteristics and schools in DCPS differ on 
even more characteristics across VAM quartiles. 
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• In Memphis, the schools that are not eligible for EPIC are systematically 
different in measures of school and student characteristics from schools that are 
eligible for EPIC. 

• In the charter school consortium, school and student characteristics differ 
substantially by location. 

These findings provide a foundation for understanding the schools included in the 
evaluation of EPIC. They confirm that the schools that received EPIC awards performed 
well on other measures of school performance. For Memphis and the charter school 
consortium, the results suggest that differences in school VAM performance might not be 
driven by differences in race, income, or English proficiency (however, we cannot mle out 
this possibility in DCPS). Finally, the results suggest that the Memphis schools ineligible for 
EPIC might not be an appropriate comparison group for the evaluation of EPIC incentive 
awards there, although MPR staff plan to conduct additional analyses of the data before 
making a determination on this issue. 



I: Introduction 




Chapter II 



Memphis City Schools 
Baseline Statistics 



N ew Leaders for New Schools began working with Memphis City Schools (MCS) in 
the summer of 2006 to establish the Effective Practice Incentive Community 
(EPIC) program, and piloted EPIC in Memphis during the 2007 / 2008 school year 
(Year 1). New Leaders presented the first awards in December 2007, which were 
based on value-added performance for the 2006/2007 school year as calculated by MPR. In 
this chapter, we begin with an overview of EPIC in Memphis and proposed changes for 
2008/2009 (Year 2). Next, we present our findings on MCS baseline characteristics, which 
include information on school performance measures, as well as school, teacher, and student 
characteristics. We present these characteristics by eligibility status, award status, and 
performance in MPR’s value-added model (VAM). 



Overview of EPIC in MCS 

To be eligible for an EPIC award, a school (1) must have at least 50 percent of the 
student body eligible for free or reduced price lunch (FRPL), (2) cannot be a charter school, 
and (3) cannot be participating in either the Fresh Start or Striving School programs. 11 
These criteria resulted in 148 eligible and 35 ineligible schools for the 2007/2008 school year 
and 139 eligible and 39 ineligible schools for the 2008/2009 school year. 12 



11 The Fresh Start and Striving School programs focus on improving student achievement in low- 
performing schools and include a pay-for-performance component for school staff. For more detail on these 
programs, see Cody et al. (2009). 

12 Since the introduction of EPIC in Memphis, there have been several changes in eligibility criteria. We 
recently verified schools’ Year 2 eligibility status with New Leaders. The primary differences between the 
number of eligible/ineligible schools in Year 2 and Year 1 are due to school closings and changes in eligibility 
criteria. Also, since the purpose of this report is to provide baseline information on schools so as to examine 
the impact of being eligible for an EPIC award in 2008/2009, this chapter identifies schools’ eligibility by their 

(continued) 
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In December 2007, EPIC presented awards to 17 schools based on their value-added 
scores for the 2006/2007 school year, as calculated by MPR. NLNS presented 5 schools 
with Gold-Gain awards and 12 schools with Silver-Gain awards. The recipients of the Gold- 
Gain awards had the greatest value-added scores. The 12 Silver-Gain schools ranked just 
below the gold schools in their value-added scores. In Memphis, after a school is nominated 
for an award, at least 80 percent of its teachers must vote to accept the award for the school 
to receive it. Five of the Memphis schools offered awards in December 2007 chose to 
decline them. Pardy in response to information obtained from this pilot year from school 
staff and administrators concerning the implementation of EPIC, NLNS is proposing 
several changes for Year 2, as explained below. 

All principals, assistant principals, and instmctional staff at award-winning schools 
receive a financial reward; however, the amount varies by position and by award status (gold 
or silver). In 2007 the principals and assistant principals received much larger awards than 
the teachers (see Table II. 1). In response to feedback concerning the discrepancies in value 
by position, New Leaders proposes to reduce the awards to principals and assistant 
principals, and increase awards to instmctional staff for fall 2008. 



Table 11.1. Memphis Award Amounts 







Gold 






Silver 




Award Announcement 
Dates 


Principals 


Assistant 

Principals 


Teachers 


Principals 


Assistant 

Principals 


Teachers 


Fall 2007 


$15,000 


$10,000 


$1,000 


$10,000 


$7,500 


$1,000 


Proposed for Fall 2008 


$10,000 


$6,750 


$2,500 


$7,500 


$5,000 


$2,500 



In addition to school-level awards, the EPIC initiative in Memphis also is designed to 
include “Spotlight Teacher” awards; these are to be given to a select group of teachers at 
award-winning schools. These teachers will be selected by NLNS site visit teams, based in 
part on the value-added scores of these teachers and in part on other criteria collected from 
their schools. Initially, New Leaders planned to award the first Spotlight Teacher awards 
during the 2007/2008 school year. New Leaders believes that successful implementation of 
the Spotlight Teacher awards — a potentially controversial component of the EPIC 
initiative — requires the full support and understanding of school leadership. Consequently 
these awards were postponed until 2008/2009, and New Leaders has asked that ED 
consider postponing them once more until 2009/2010. 



( continued) 

2008/2009 eligibility status. Although there are 139 eligible schools in Year 2, we have information on only 138 
schools because one school opened in 2008/2009. 
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Baseline Characteristics 

For the remainder of this chapter, we present baseline characteristics of MCS according 
to EPIC eligibility, award status, and VAM performance (whether in the top, middle two, or 
bottom quartiles). 13 The baseline characteristics comprise three broad categories: (1) school 
performance measures, (2) school characteristics, and (3) student characteristics. 

Data on school performance and school characteristics were obtained from publicly 
available data at the school level. We were successful in obtaining complete information on 
almost all schools; however, for a few variables, we were missing data for a small proportion 
of schools. To estimate student characteristics, we created a data set of student -level data for 
each school for those students who were included in the VAM data set. 

The following characteristics were measured: 

• School Performance Measures. Percentage of schools passing adequate yearly 
progress (AYP); percentage of schools identified as being in need of 
improvement, or SINI (NCLB classifies schools that fail AYP in consecutive 
years and subsequently have not passed AYP for two consecutive years as 
“schools in need of improvement”); percentage of students proficient in 
English/Language Arts (ELA); percentage proficient in math; and the school’s 
promotion rate. These data are all for the 2006-2007 school year. 

• School Characteristics. Enrollment, student/ teacher ratio, staff size, teacher 
ethnicity, and percentage of teachers with a Masters or PhD. For the breakdown 
by eligibility status, we also include school level (elementary, middle, and high 
schools). These data are from the most recent year of data we could get from 
the Common Core of Data (2005-2006). 

• Student Characteristics. Student ethnicity, percentage of students eligible for 
FRPL, percentage of LEP students, and percentage of special education 
students. These data are used to estimate our VAM scores and are for the 2006- 
2007 school year. 

Baseline Characteristics by EPIC Eligibility Status 

As explained in the design report (Cody et al. 2009), we hope to estimate the incentive 
impact of EPIC awards by comparing student achievement growth between schools eligible 
and those ineligible for an award. All else being equal, eligibility for an EPIC award can 
provide additional motivation for school staff to increase student achievement. If so, we 
would expect the growth in student test scores to be greater for EPIC-eligible schools than 
for the ineligible schools. 



13 VAM performance was measured for the 2006-2007 school year. See Booker and Isenberg (2008) for 
details. 
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E PIC-Eligible Versus EPIC-ineligible Schools 

Table II.2 compares baseline characteristics between the 138 EPIC-eligible schools and 
the 39 EPIC-ineligible schools. Eligible and ineligible schools are meaningfully and 
significantly different on many of our measured characteristics. First, the distribution of 
schools by school level is quite different for eligible versus ineligible schools. More than 70 
percent of eligible schools are elementary schools, while less than 20 percent of ineligible 
schools are elementary. 14 Schools also significantly differ by school performance measures, 
with eligible schools performing better on average. For instance, 80 percent of eligible 
schools passed AYP, yet less than 60 percent of ineligible schools passed AYP. Similarly, a 
much higher percentage of ineligible schools are “in need of improvement” (69 percent) 
compared to eligible schools (15 percent). 

Eligible and ineligible schools also significantly differ by several school and student 
characteristics. Ineligible schools average almost 9 percent higher enrollment (720 students 
versus 661) and have a higher student/teacher ratio. Eligible schools, on average, have a 
significantly higher percentage of black students than ineligible (92 versus 85 percent) and a 
higher percentage of students eligible for FRPL (86 percent compared to 72 percent). In 
terms of teacher characteristics, we find no significant difference between eligible and 
ineligible schools. 

In the next section, we separate ineligible schools by reason of ineligibility to examine 
whether there might be a subset of these schools more comparable to our eligible schools. 

EPIC-Eligible versus EPIC-ineligible Schools by Reason of Ineligibility 

A school may be ineligible for an EPIC award for several reasons. We can separate 
EPIC-ineligible schools into six categories: (1) alternative schools, (2) high-income schools, 
(3) charter schools, (4) “special” schools, 15 (5) Fresh Start schools, and (6) Striving schools. 16 



14 Since the distribution by school level differed dramatically between eligible and ineligible schools, we 
also examined baseline characteristics excluding elementary schools. However, we found very similar results 
both in the magnitude of the differences and the significance. 

15 At this time, we do not have specific information on what determines a school being categorized as 
“alternative” or “special.” 

16 Each school was assigned to only one category, although a few schools qualified for more than one 
classification. For instance, several schools are both Fresh Start and Striving schools. Since Fresh Start schools 
were ineligible in Year 1 and Striving schools became ineligible only in Year 2, if a school fell into both 
categories, it was classified as a Fresh Start school. Also, schools were classified as ineligible because they were 
high income only if they were not ineligible for any other reason. 



II: Memphis City Schools Baseline Statistics 




21 



Table 11.2. Baseline Statistics by EPIC Eligibility, Memphis City Schools 



Category 


Eligible 


Ineligible 


School Level (%) 


Elementary schools 


71.7 


18.4“* 


Middle schools 


15.9 


36.8“* 


High schools 


12.3 


44 . 7 *** 


School Performance 


AYP pass (%) 


80.3 


57.1*“ 


SINI (%) 


15.4 


69.2*“ 


Promotion rate 


93.1 


90.3 


School Characteristics 


Enrollment 


660.7 


719.5 


Student-teacher ratio 


17.0 


19.4*** 


Staff size 


37.7 


39.9 


Teacher ethnicity (%) 


White 


31.3 


33.5 


Black 


66.9 


64.9 


Hispanic 


0.4 


0.6 


Master’s or Ph.D. (%) 


60.6 


63.2 


Student Characteristics (%) 
Ethnicity 


Black 


92.0 


85.1“ 


Hispanic 


4.0 


2.9 


FRPL 


86.1 


71.8*** 


LEP 


3.2 


1.6 


Special education 


11.0 


10.3 


Number of Schools 


138 


39 



‘Significantly different from the eligible schools’ value at the 10% level. 

‘‘Significantly different from the eligible schools’ value at the 5% level. 

‘“Significantly different from the eligible schools’ value at the 1% level. 

Notes: Only Memphis schools eligible for an EPIC award in 2008-2009 are included in these 
statistics. 

School performance measures and school characteristics were obtained from publicly 
available data (Common Core of Data for 2005-2006 school year). Student 
characteristics were averaged across all students included in the VAM (2006-2007 school 
year). 
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As indicated in Table II. 3, the characteristics of ineligible schools vary among these six 
ineligibility classifications. The alternative schools are either middle or high schools in need 
of improvement and have the highest percentage of special education students, at almost 23 
percent. The next highest, Fresh Start schools, have an average of less than 16 percent 
special education students. The 10 high-income schools not only have the lowest percentage 
of students eligible for FRPL (as expected by the category’s definition) but also the highest 
percentage of students who are proficient in ELA and math. Charter schools have relatively 
low enrollment, a high percentage of teachers with a master’s degree or a Ph.D., and a low 
percentage of special education students. The three special schools are all high schools; they 
have relatively small enrollments and the lowest percentage of special education students. 
Both Fresh Start and Striving schools stand out in terms of doing poorly on the performance 
measures. 

Comparing each of these ineligibility categories with eligible schools, we continue to see 
significant and meaningful differences. For all categories, the distribution by school level 
continues to be significantly different than eligible schools. Except for charter and special 
schools, for which we lack proficiency information, each category of ineligible schools is 
significantly different from eligible schools on performance measures. High-income schools 
perform significantly better, while alternative, Fresh Start, and Striving schools perform 
significantly worse. 17 High -income schools are significantly different and are the least similar 
to ineligible schools in terms of teacher characteristics, percentage of students who are black, 
and percentage of students eligible for FRPL. Charter schools are significantly different on 
school characteristics (both size and teacher characteristics), as well as the percentage of 
students eligible for FRPL and the percentage of special education students. 

Fresh Start and Striving schools are the most similar to eligible schools on some 
characteristics, yet are significantly different in a number of other ways (in addition to the 
performance measures). Fresh Start schools are the most similar in terms of school size and 
also are similar in terms of teacher ethnicity. However, Fresh Start schools have a 
significantly lower percentage of teachers with a master’s degree or a Ph.D. (47 percent) than 
eligible schools (61 percent). Striving schools are the most similar and are not statistically 
different from eligible schools in terms of teacher and student characteristics. 

Baseline Characteristics by Award Status 

Table II.4 shows the baseline characteristics for the 138 EPIC-eligible schools by award 
status — award recipients, non-award schools, and award decliners. Twelve elementary, three 
middle, and two high schools received awards. Three elementary and two high schools 
declined an award offer. Since one award recipient and one declining school became 
ineligible in Year 2, the table includes only 16 award recipients and 4 declining schools. 



17 The Fresh Start and Striving programs target the lowest performing schools, so we would expect to 
find this reflected in our baseline performance characteristics. 
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Table 11.3. Baseline Statistics for Eligible vs. Ineligible Schools by Detailed Categories, Memphis City Schools 



Category 


Eligible Schools 


Alternative 


High Income 


Special 


Charter 


Fresh Start 


Striving 


School Level (%) 


Elementary schools 


71.7 


0.0"* 


40.0" 


0.0*" 


25.0*" 


0.0*" 


11.1"* 


Middle schools 


15.9 


33.3 


30.0 


0.0 


50.0" 


80.0*" 


22.2 


High schools 


12.3 


66.7" 


30.0 


100.0*" 


25.0 


20.0 


66.7*" 


School Performance 


AYP pass (%) 


80.3 


0.0"* 


80.0 


n.a. 


100.0 


20.0*" 


55.6* 


SINI (%) 


15.4 


100.0*" 


22.2 


n.a. 


0.0 


100.0*" 


100.0*** 


Student proficiency (%) 
ELA proficient 


83.7 


71.0*" 


95.0*** 


n.a. 


n.a. 


75.2*" 


82.8 


Math proficient 


80.6 


44.3*" 


89.2*** 


n.a. 


n.a. 


69.4*" 


67.7*** 


Promotion rate 


93.1 


92.0 


95.9 


99.5 


97.5 


88.8 


78.2*** 


School Characteristics 


Enrollment 


660.7 


193.5* 


1,120.7"* 


141.7" 


183.0*" 


773.8 


1,029.9*** 


Student-teacher ratio 


17.0 


10.5* 


18.7 


14.6 


25.7"* 


18.2 


19.2 


Staff size 


37.7 


16.5* 


58.1"* 


1 1 .7" 


9.3*" 


42.8 


53.2*** 


Teacher ethnicity (%) 
White 


31.3 


25.5 


63.4*** 


11.4* 


10.6" 


21.0 


28.9 


Black 


66.9 


74.5 


33.8*** 


83.6 


89.2*** 


78.4 


70.0 


Hispanic 


0.4 


0.0 


1.6*** 


0.0 


0.0 


0.2 


0.2 


Master’s or Ph.D. (%) 


60.6 


69.0 


66.1 


70.7 


78.3*" 


46.9" 


56.7 


Student Characteristics (%) 
Ethnicity 
Black 


92.0 


99.4 


52.4*" 


99.0 


98.1 


98.2 


91.4 


Hispanic 


4.0 


0.5 


4.2 


1.0 


1.7 


1.6 


4.8 


FRPL 


86.1 


88.9 


33.9*** 


71.0" 


78.2" 


93.1 


89.0 


LEP 


3.2 


0.0 


2.5 


0.0 


1.0 


0.4 


3.1 


Special education 


11.0 


22.6*" 


7.2*** 


3.4*" 


6.8*" 


15.6" 


10.7 


Number of Schools 


138 


4 


10 


3 


8 


5 


9 



'Significantly different from eligible schools at the 10% level. 

"Significantly different from eligible schools at the 5% level. 

"'Significantly different from eligible schools at the 1% level. 

Notes: Based on Year 2 EPIC eligibility status. 

School performance measures and school characteristics were obtained from publicly available data (Common Core of Data for 2005-2006 school year). Student 
characteristics were averaged across all students included in the VAM model (2006-2007 school year). 



n.a. = not applicable. 
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Table 11.4. Baseline Statistics by Award Status, Memphis City Schools 



Category 


Award 

Recipients 


Non-Award 

Schools 


Declined 

Award 


School Performance 
AYP pass (%) 


93.8 


78.0 


100.0 


SINI (%) 


6.3 


16.2 


33.3 


Student proficiency (%) 
ELA proficient 


86.2* 


83.3 


87.3 


Math proficient 


85.7“ 


79.8 


84.5 


Promotion rate 


94.5 


92.9 


92.3 


School Characteristics 
Enrollment 


580.8 


671.9 


649.8 


Student-teacher ratio 


16.2 


17.1 


17.7 


Staff size 


34.0 


38.3 


35.4 


Teacher ethnicity (%) 
White 


28.5 


31.7 


32.0 


Black 


70.1 


66.6 


64.5 


Hispanic 


0.1 


0.4 


1.6* 


Master’s or Ph.D. (%) 


65.3 


60.1 


54.2 


Student Characteristics (%) 
Ethnicity 
Black 


92.8 


91.8 


93.8 


Hispanic 


3.9 


4.0 


3.9 


FRPL 


86.7 


85.8 


90.2 


LEP 


3.2 


3.2 


3.0 


Special education 


10.9 


10.9 


13.1 


Number of Schools 


16 


118 


4 



‘Significantly different from non-award schools at the 10% level. 

“Significantly different from non-award schools at the 5% level. 

“‘Significantly different from non-award schools at the 1% level. 

Notes: Only Memphis schools eligible for an EPIC award in 2008-2009 are included in these 
statistics. 

School performance measures and school characteristics were obtained from publicly 
available data (Common Core of Data for 2005-2006 school year). Student 
characteristics were averaged across all students included in the VAM model (2006-2007 
school year). 
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• Performance Characteristics. As might be expected, award-winning schools 
tend to perform better on school performance characteristics than non-award 
schools, although the differences are not large. For example, 86 percent of the 
award winners’ students are proficient in both ELA and math, compared to 83 
percent and 80 percent, respectively, for the non-award winners. 18 Although 
none of the other performance measures are significantly different, both the 
award winners and decliners tend to perform better than non-award schools. 19 

• School and Student Characteristics. We examined 12 non-performance 
characteristics and made two comparisons for each (award winners vs. non- 
award winners and decliners vs. non- award winners). Only one of these 24 
comparisons is statistically significant at the 10 percent level (decliners vs. non- 
award winners on percentage of Hispanic staff). This suggests that there are not 
clear differences in these other characteristics by award status. 

Overall, only two of the 17 comparisons between award recipients and non- award 
schools are significantly different, and those characteristics relate to school performance. 
Only one comparison between the declining and non-award schools is statistically different. 
Furthermore, many of the mean characteristics are very similar in magnitude across the three 
categories, and typically there is no consistent pattern by award status. 

Baseline Characteristics by VAM Quartile Scores 

Table II. 5 compares characteristics between schools based on their performance in 
MPR’s value-added model (VAM). To create these categories, schools first were ranked into 
quartiles by school level. The middle two quartiles were combined. Therefore, the top and 
bottom quartile categories based on Year 1 eligibility consisted of 25 elementary, 6 middle, 
and 5 high schools, while the middle category included 51 elementary, 12 middle, and 10 
high schools. However, Table II. 5 includes only schools that are EPIC-eligible in Year 2; 
since some schools that were EPIC-eligible in Year 1 no longer are eligible in Year 2, the 
statistics are based on fewer schools than in the original quartiles. 20 All significance tests are 
in relation to the “middle quartiles” category. 

• Performance Characteristics. Schools in the bottom quartile perform worse 
than those in the middle quartiles on three of the five performance measures 
considered (AYP, percentage proficient in ELA, and percentage proficient in 
math). Also, schools in the top quartile perform significantly better than schools 
in the middle quartiles on math proficiency (85 percent versus 81 percent). 



18 There were very few decliners, which may explain why the differences between the decliners and non- 
award winners were not statistically significant. 

19 One notable exception is that decliners have a higher percentage SINI (1 out of 3 with data) compared 
to the non-award winners, yet again the difference is not statistically significant. 

20 See Appendix Tables A.4-A.6 for the breakdown of the number of schools included in the VAM 
comparisons by school level. 
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None of the other performance differences is statistically significant but, in 
almost all cases, schools in the top quartile perform better than schools in the 
middle quartiles and those in the middle perform better than schools in the 
lowest quartile. The percentage of schools passing AYP differs quite a bit across 
quartiles: 91 percent for the top quartile, 83 percent for the middle quartiles, and 
only 65 percent for the bottom quartile. Since both a school’s AYP status and its 
performance in the VAM model capture information on student performance, 
we could expect to see this type of relationship. In terms of percent proficient, 
the top and middle quartiles are similar — on average, 81-86 percent of their 
students are proficient. However, the bottom quartile schools average less than 
80 percent proficient, and both measures are significantly different than the 
middle quartiles. 



Table 11.5. Baseline Statistics by VAM Quartile Scores, Memphis City Schools 



Category 


Top Quartile 


Middle Quartile 


Bottom Quartile 


School Performance 
AYP pass (%) 


91.2 


82.6 


64.7** 


SINI (%) 


9.1 


15.9 


20.6 


Student proficiency (%) 
ELA proficient 


86.4 


84.5 


79.4*** 


Math proficient 


85.4** 


81.2 


74.3*** 


Promotion rate 


94.7 


92.5 


92.5 


School Characteristics 
Enrollment 


639.1 


685.5 


632.5 


Student-teacher ratio 


17.0 


17.1 


16.8 


Staff size 


36.3 


39.1 


36.3 


Teacher ethnicity (%) 
White 


31.4 


32.0 


29.9 


Black 


67.2 


65.8 


68.8 


Hispanic 


0.3 


0.5 


0.6 


Master’s or Ph.D. (%) 


62.6 


59.7 


60.2 


Student Characteristics (%) 
Ethnicity 
Black 


93.6 


90.2 


93.9 


Hispanic 


3.6 


4.6 


3.2* 


FRPL 


85.4 


84.9 


89.1 


LEP 


2.9 


3.5 


2.7 


Special education 


11.0 


10.9 


11.2 


Number of Schools 


35 


69 


34 



‘Significantly different from middle quartile schools at the 10% level. 

“Significantly different from quartile schools at the 5% level. 

‘“Significantly different from middle quartile schools at the 1% level. 

Notes: Only Memphis schools eligible for an EPIC award in 2008-2009 are included in these statistics. 

School performance measures and school characteristics were obtained from publicly available 
data (Common Core of Data for 2005-2006 school year). Student characteristics were averaged 
across all students included in the VAM (2006-2007 school year). VAM quartiles were based on 
2006-2007 school year performance. 
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• School Characteristics. School characteristics are very similar across all 
quartiles, and none is significantly different. Enrollment ranges, on average, 
from 633 students for the bottom quartile to 686 students for the middle 
quartiles; the student/teacher ratio is nearly identical at about 17 students per 
teacher; the percentage of white teachers is ranges from 30 to 32 percent; and 
the percentage of teachers with a Masters or PhD ranges from 60 to 63 percent. 

• Student Characteristics. Student characteristics are also similar across 
quartiles. For instance, the percentage of LEP is less than 4 percent for all 
quartiles, and the percentage of special education students is approximately 11 
percent for all quartiles. The only significant difference is the percentage of 
students eligible for FRPL for the bottom quartile, and even that difference is 
not large (89 percent for the bottom quartile, compared to 85 percent for the 
middle quartiles). 

Appendix tables A.4-A.6 show these comparisons by VAM quartiles separately by 
school level (elementary, middle and high school). Those tables show similar findings to 
those in Table II. 5, with a few exceptions. In general, differences in performance measures 
often become larger (and significant) when we compare VAM quartiles separately by school 
level. 

Conclusion 

This chapter compares baseline performance, school, and student characteristics by 
award status, VAM performance, and eligibility status. We find similar results when we 
compare schools by award status and VAM performance. Award-winning schools and 
schools that perform well in terms of the VAM model (clearly related, but not an identical 
groups of schools), tend to perform better on school performance measures than 
comparison schools, as we would expect. However, we find that schools are fairly similar in 
terms of school and student characteristics, and often are not statistically different when we 
compare these characteristics by award or VAM performance categories. In terms of EPIC, 
there are no obvious correlations between school or student characteristics and which 
schools tend to do well in MPR’s value-added model, and so are more likely to win an award. 

However, our comparisons between EPIC-eligible and EPIC -ineligible schools show 
numerous differences on baseline characteristics, and how the schools differ depends on the 
reason for their ineligibility. This suggests that we will need to proceed with caution when 
using this comparison group to estimate impacts of EPIC, and that a subset of ineligible 
schools may be more appropriate to use for a comparison group. 
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Chapter III 



Charter Schools Baseline Statistics 



I n 2007, NLNS piloted the EPIC program with participation from charter schools in 18 
states and the District of Columbia. 21 These schools were recruited into the EPIC 
program as early as 2006 by NLNS with help from the New Schools Venture Fund, a 
philanthropic nonprofit entity that invests in education endeavors, particularly charter 
schools. We begin this chapter with an overview of EPIC in charter schools, including 
eligibility requirements, Year 1 performance awards, and proposed changes for 2008-2009. 
Next, we discuss plans to evaluate the impact of EPIC incentives on charter school student 
outcomes. Finally, we present our findings on charter school baseline (Year 1) 
characteristics, including information on school performance measures, as well as school, 
teacher, and student characteristics. We present these characteristics by award status, VAM 
performance, and school location. 



Overview of EPIC in Charter Schools 

To be eligible for an EPIC award, at least 30 percent of a charter school’s student body 
must be eligible for free and reduced price lunch (FRPL), sign a Memorandum of 
Understanding (MOU) with NLNS, and provide MPR with the student test scores and 
demographic data needed to estimate a value-added model. Although more than 1 30 charter 

schools expressed interest in EPIC in Year 1, only 97 schools completed the necessary 

22 

steps. 

NLNS announced the first awards for charter schools in March 2008. Schools were 
chosen for awards based on value-added measures of their performance during the 2006- 
2007 school year, as calculated by MPR (Booker et al. 2008). The award-winning charter 



21 States that participated in Year 1 of the EPIC program were: AZ, CA, CO, DC, FL, IL, IN, MA, MD, 
MI, MO, NC, NM, NY, OH, PA, TN, TX, and WI. 

22 Two schools were close to being eligible but did not provide data on enough students to qualify. 
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schools were from nine states and the District of Columbia. 21 Schools with the highest 
value-added scores in their school-level (elementary, middle, or high) received Gold-Gain 
awards, and the schools in the next tier of value-added scores received Silver-Gain awards. 
Schools that serve multiple grade levels (e.g., both elementary and middle) were eligible for 
awards in all grade levels but could receive only one award. NLNS granted seven Gold-Gain 
awards, which included three elementary schools, three middle schools and one high school. 
Gold-gain principals were awarded $20,000, assistant principals were awarded $15,000, and 
all instmctional staff were awarded $1,500. Fifteen schools (six elementary, six middle 
schools, and three high schools) received Silver-Gain awards of $15,000, $10,000, and $750 
for each staff designation, respectively. In the spring of 2008, NLNS visited Gold-Gain 
charter schools to identify promising practices and gauge educators’ reaction to EPIC. 

NLNS is in the process of recruiting additional charter schools to be eligible for awards 
in 2008-2009. The second round of awards will be given out for both school and individual 
teacher performance. Teachers in award-winning schools with the highest teacher-level 
value-added scores (called “Spotlight Teachers”) each will earn an additional $10,000 award. 
To be eligible for participation in Year 2, schools must submit both student -level test scores 
and demographic data (as was required in Year 1) and student-teacher linking data to aid in 
the identification of Spotlight Teachers. It is unclear at this time how many additional 
schools may be added to the 97-school consortium. Also, it appears that some of the original 
97 schools may choose not to participate. 

Evaluation of EPIC in Charter Schools 

New Leaders has asked MPR to investigate the possibility of evaluating the impact of 
EPIC incentives on student outcomes in charter schools as we are doing in Memphis. To do 
so, we propose to estimate a difference-in-differences model to compare the gains in 
students’ scores between schools that participate in the EPIC program (the “treatment” 
schools) to gains of schools that do not participate (the “comparison” schools). 24 If 
participating schools have a greater incentive to improve students’ scores because of the 
potential for an award, we would expect the gain in scores from these schools to be greater 
than the gain in students’ scores from non-participating schools, when all else is equal. 



23 Because the VAM includes test scores for multiple grades, subjects, and years, as well as scores for 
different states that administer different exams, the scores must be standardized so that they fit comparable 
scales. MPR transforms the test scores by subtracting from each student’s score the statewide mean for that 
subject, grade, and year, and dividing by the statewide standard deviation for the same variables. This yields a 
standardized score that equates each student to the average student in the state, and is comparable across 
schools within each state. To allow comparison of test scores across different states, MPR adjusts student 
scores using state average scores and standard deviations from the National Assessment of Educational 
Progress (NAEP). 

24 Although in theory all charter schools serving at least 30 percent FRPL students are eligible to 
participate in EPIC, only those that submit data and sign an MOU are eligible for awards. 



Ill: Charter Schools Baseline Statistics 




31 



Although the methodology is straightforward, we now have value-added data only for 
the participant charter schools. Data thus are needed from non-participant schools to create 
a comparison group. We are in the process of requesting data for this from several states and 
districts. Additionally, the research design assumes that the staff at charter schools that 
participated in Year 1 know about the EPIC incentives. Feedback from New Leaders, 
however, suggests that some principals may have informed staff about the EPIC incentives 
only if the school received an award. We plan to conduct a teacher survey in the spring of 
2009 that will ask about awareness of EPIC, thus providing information on whether or not 
an evaluation of EPIC in charter schools is feasible. 

Baseline Characteristics of Year 1 EPIC Participants 

This section presents a comparison of student and school baseline characteristics in 
charter schools that were eligible for EPIC awards in Year 1. The characteristics fall into 
three broad categories: (1) school performance measures, (2) school characteristics, and (3) 
student characteristics. 

Data on school performance and school characteristics were obtained from publicly 
available data at the school level. We were successful in obtaining complete information on 
all schools in CA, IL, and DC; however, for a few variables we were missing data for several 
schools in other states. To estimate student characteristics, we created a school -level data set 
based on student-level data for students at the school who were included in the VAM data 
set. 



The characteristics measured are: 

• School Performance Measures. Percentage of schools passing AYP, 
percentage of schools identified as in need of improvement (“SINI”). These 
data are from the 2006-2007 school year when possible and from 2005-2006 
otherwise. 

• School Characteristics. Enrollment, student/teacher ratio, staff size, years 
operating. These data are from the 2006-2007 school year when possible and 
from 2005-2006 otherwise. 

• Student Characteristics. Student ethnicity, percentage of students eligible for 
FRPL, percentage of students who are limited English proficient (LEP), and 
percentage of students identified as in special education. These data are based 
on the students used to calculate the VAM scores and are for the 2006-2007 
school year. 

We use these comparisons to determine if schools systematically differed by (1) whether 
or not they won an EPIC award, (2) performance in the value-added model,"" and 

25 VAM performance was measured for the 2006-2007 school year. See Isenberg, Booker, and Chaplin 
(2008) for details. 
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(3) location. 26 In general, we find that schools that did well in Year 1 (1) had lower student 
enrollment, (2) served more FRPL eligible students, and (3) served more students identified 
as special education than schools that did not do well. Otherwise, participating charter 
schools look very similar across VAM quartiles and award status. Most variation in school 
and student characteristics occurs across school location. 

Baseline Characteristics by Award Status 

Table III. 1 shows the baseline characteristics for the 97 charter schools by Year 1 award 
status — award winners vs. those that did not win an award. 

• Performance and School Characteristics. Charter schools that won awards 
and those that did not are very similar in terms of performance and school 
characteristics. The majority of schools made Adequate Yearly Progress (AYP) 
in the previous year, and approximately one-third of schools have been 
designated “in need of improvement” (SINI status) under NCLB for failing 
AYP in two consecutive years. Ninety-six percent of all schools have been 
operating for at least three years, which some literature identifies as a turning 
point for charter success. Only student enrollment significantly differs across 
award winners and non-award schools. Non-award winning schools are 51 
percent larger than award winners, serving 456 versus 302 students on average. 

• Student Characteristics. Of the five comparisons made between students at 
award winning and non-award winning schools, two are significantly different. 

At award winning schools, 75 percent of students are eligible for free and 
reduced price lunch and 16 percent are identified as special education, compared 
to just 63 percent and 9 percent, respectively at schools that did not win awards. 
These findings suggest that award winning schools serve more at-risk students 
than do non-award winners. On the other hand, the racial and ethnic 
composition of both groups is similar. On average, at both award winning and 
non-award winning schools, approximately half of the students are black and 
one-quarter are Hispanic. 

Overall, award winning and non-award winning charter schools are similar in AYP 
performance, the size of the teaching staff, years operating, and student ethnicity. Award 
recipients are significantly smaller in size, a school characteristic that some literature has 
suggested is correlated with school performance. Significant differences in student 
characteristics are limited to FRPL eligibility and special education status, both significant at 
the .05 level. 



26 Once a comparison group is identified, baseline characteristics of participating charter schools will be 
compared to those of non-participating charter schools. 
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Table 111.1. Baseline Statistics by Award Status, Charter Schools 



Category 


Award Recipients 


Non-Award 

Schools 


School Performance (%) 


AYP pass 


65.0 


52.1 


SINI 


30.0 


38.0 


School Characteristics 


Enrollment 


301.7 


456.2“ 


Student-teacher ratio 


16.2 


17.4 


Staff size 


24.1 


28.2 


Years operating 
2 years or fewer 


0.0 


5.4 


3-5 years 


36.4 


35.1 


6 years or more 


63.6 


59.5 


Student Characteristics (%) 
Ethnicity 
Black 


49.7 


50.4 


Hispanic 


27.4 


24.7 


FRPL 


74.8 


63.1“ 


LEP 


10.0 


6.1 


Special education 


15.5 


9.2“ 


Number of Schools 


22 


75 



‘Significantly different from the award recipients at the 1 0% level. 

“Significantly different from award recipients at the 5% level. 

“‘Significantly different from the award recipients at the 1% level. 

Note: School performance measures and characteristics were obtained from publicly available 

data — 2006-2007 when possible and 2005-2006 otherwise. Student characteristics were 
averaged across all students in the VAM (2006-2007 school year). 



Baseline Characteristics by VAM Quartile Scores 

Table III.2 compares characteristics between schools based on their performance in 
MPR’s value-added model (VAM). To create these categories, schools first were ranked into 
quartiles by school level. 27 Next, the middle two quartiles were combined. When the school- 
level rankings are aggregated, the top and bottom quartile categories consist of 23 schools 
each, while the middle category has 51 schools. This breakdown of school and student 
characteristics enables us to examine how schools in the top quartile (which includes only 
one non-award winning school) differ from schools that rank in the middle two quartiles and 



27 Elementary schools serve grades 2 through 6 (or any subset of those grades), middle schools serve 
grades 7 and 8, and high schools serve 9th grade and above. Although charter schools serving multiple school 
levels are eligible for multiple award pools, schools were assigned to their highest school level to make these 
baseline comparisons. Schools that won an award at other than their highest level served were assigned to the 
award-winning school level. 
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those that rank in the bottom quartile. All statistical tests are conducted between the top or 
bottom VAM quartile and the middle two quartiles. 



Table 111.2. Baseline Statistics by VAM Quartile Scores, Charter Schools 



Category 


Top Quartile 


Middle Quartile 


Bottom Quartile 


School Performance (%) 


AYP pass 


68.2 


57.1 


35.0 


SINI 


36.4 


34.7 


40.0 


School Characteristics 


Enrollment 


312.9 


429.3 


511.3 


Student-teacher ratio 


16.8 


16.7 


18.2 


Staff size 


23.5 


27.1 


31.3 


Years operating 
2 years or fewer 


4.3 


3.9 


4.5 


3-5 years 


34.8 


35.3 


36.4 


6 years or more 


60.9 


60.8 


59.1 


Student Characteristics (%) 
Ethnicity 
Black 


51.2 


48.8 


52.4 


Hispanic 


32.0 


24.5 


20.5 


FRPL 


74.9* 


63.9 


60.4 


LEP 


10.7 


6.6 


4.1 


Special education 


14.6* 


9.4 


9.4 


Number of Schools 


23 


51 


23 



‘Significantly different from middle quartile schools at the 10% level. 

“Significantly different from middle quartile schools at the 5% level. 

“‘Significantly different from middle quartile schools at the 1% level. 

Note: School performance measures and school characteristics were obtained from publicly 

available data--2006-2007 when possible and 2005-2006 otherwise. Student 
characteristics were averaged across all students included in the VAM (2006-2007 school 
year). VAM quartiles were based on 2006-2007 school year performance. 

• Performance Characteristics. Neither of the performance measures (AYP or 
SINI) differs significantly by VAM status, although AYP performance ranges 
from a pass rate of 68 percent for top quartile schools to 35 percent for bottom 
quartile schools. SINI status varies very little across quartiles, suggesting that 
some top quartile SINI schools may be making progress towards moving from a 
status of “in need of improvement” to “good standing.” 

• School Characteristics. Of the 12 comparisons made across school 
characteristics, no differences are statistically significant. However, top quartile 
schools serve fewer students and have somewhat fewer teachers. Enrollment 
ranges, on average, from 511 students for the bottom quartile to 313 students 
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for the top quartile, and the size of the teaching staff ranges from 31 teachers 
for bottom quartile schools to 24 for those in the top. 

• Student Characteristics. Of the 10 comparisons made across student 
characteristics, only two are statistically significant at the 10 percent level. This 
result might be due to chance. Schools in the top quartile serve significantly 
more FRPL eligible students and students identified as in special education than 
do schools in the middle two quartiles (75 vs. 64 percent and 15 vs. nine 
percent, respectively). 

In the Appendix, Tables A.7-A.9 show these comparisons by VAM quartiles separately 
for elementary, middle, and high schools. When we disaggregate these results by school level, 
the differences between the top quartile schools and the middle two quartiles are seldom 
statistically significant. There are 13 comparisons for each school type, for a total of 39 
comparisons. Only three of these are statistically significant at the 10 percent level, a result 
that easily could be due to chance. A comparison of the bottom quartile with the middle two 
quartiles yields similar results for middle and high schools — 26 comparisons, with only two 
statistically significant at the 10 percent level. The picture looks somewhat different for 
elementary schools. There, the bottom quartile does seem to stand out, as only 4 of the 13 
comparisons are statistically significant. Bottom quartile elementary schools serve 
significandy more students and have more teachers and a higher student/ teacher ratio than 
do schools in the middle two quartiles. They also serve a higher percentage of black students 
than do middle quartile schools (85 percent compared to 55 percent, on average). 28 

As expected, the comparison of top to middle quartile schools closely mirrors the 
comparison of award winners to schools that did not win an award. In particular the top 
quartile stands out in terms of the fraction of students eligible for FRPL and the fraction 
with special education status, compared to the middle quartiles of schools. Although the 
bottom quartile significantly differs from the middle two quartiles on both school and 
student characteristics, it does so only by schooltievel, and mostly for elementary schools. 

Baseline Characteristics by Location 

Table III. 3 compares characteristics between schools based on their location. We chose 
to compare schools across location, since education policies, testing instmments, and general 
education environments vary by state. Schools in CA, IL, DC, and FL were separated from 
those elsewhere because these states are disproportionately represented in the sample of 
eligible EPIC schools. Statistical comparisons were made relative to all other states, which 
include AZ, CO, IN, MA, MD, MI, MO, NC, NM, NY, OH, PA, TN, TX, and WI. 



28 These results should be interpreted cautiously, as it is possible that some schools reported the total 
number of teachers, whereas others reported only full-time teachers. 
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Table 111.3. Baseline Statistics by Location, Charter Schools 



Category 


Other 

Locations 


CA 


IL 


DC 


FL 


School Performance (%) 


AYP pass 


70.0 


86.7 


43.8* 


14.3*** 


43.8* 


SINI 


26.7 


6.7 


68.8*** 


71 .4*** 


18.8 


School Characteristics 


Enrollment 


392.1 


253.8 


514.7 


464.9 


511.5 


Student-teacher ratio 


15.3 


20.9“* 


15.0 


18.6* 


18.6* 


Staff size 


25.0 


12.5“* 


35.5“ 


38.6*** 


28.9 


Years operating 
2 years or fewer 


5.6 


0.0 


6.3 


0.0 


6.3 


3-5 years 


30.6 


57.1* 


37.5 


35.7 


25.0 


6 years or more 


63.9 


42.9 


56.3 


64.3 


68.8 


Student Characteristics (%) 
Ethnicity 
Black 


48.9 


30.4* 


70.7“ 


84.8*** 


20.9*** 


Hispanic 


19.8 


58.9*“ 


22.3 


11.9 


20.9 


FRPL 


69.0 


73.0 


83.4* 


71.0 


38.3*** 


LEP 


4.7 


15.4*** 


3.5 


7.5 


7.2 


Special education 


10.3 


7.3 


14.7 


11.5 


9.6 


Number of Schools 


36 


15 


16 


14 


16 



‘Significantly different from middle quartile schools at the 10% level. 

‘‘Significantly different from middle quartile schools at the 5% level. 

‘“Significantly different from middle quartile schools at the 1% level. 

Note: School performance measures and school characteristics were obtained from publicly 

available data--2006-2007 when possible and 2005-2006 otherwise. Student 
characteristics were averaged across all students included in the VAM (2006-2007 school 
year). VAM quartiles were based on 2006-2007 school year performance. 

• School Performance. In terms of NCLB performance measures, the majority 
of participating charter schools in CA and other states made AYP in the 
previous year. In contrast, schools in DC are less likely than those in any other 
state to have made AYP — 14 percent of DC schools pass, compared to the next 
lowest pass rate of 44 percent for both IL and FL (p=.01). State 
SINIdesignations follow a similar pattern. Whereas less than 20 percent of the 
schools in CA and FL are designated SINI, 71 percent of DC schools and 69 
percent of IL schools are classified as such. Overall, CA performs the best on 
both NCLB performance measures while DC performs the worst. 

• School Characteristics. The most significant variations in school 
characteristics across location are in the total number of teachers and 
student/teacher ratios. Teaching staff range in size from an average of 13 
teachers per school in CA to 39 teachers per school in DC. These values and 
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those of IL are significantly different than other states. Similarly, CA, DC, and 
FL have significantly higher student/ teacher ratios than other states, on average. 

• Student Characteristics. Schools in CA, IL, DC, and FL significantly differ 
from schools in other states on several student characteristics. In particular, 
there is wide variation in student ethnicity. On average, the proportion of black 
students served by CA, IL, DC, and FL schools significantly differ from the 
proportion served by other states. DC schools have the highest percentage of 
black students (85 percent), while FL schools have the lowest (21 percent). Half 
of the students in other states are black. Not surprisingly, CA has the largest 
proportion of Hispanic students (59 percent) and those with LEP (15 percent). 
FRPL eligibility also differs somewhat across location. In all states except for 
FL, a majority of students qualified for FRPL — at 38 percent, FL is significantly 
different than other states, where 60 percent of students qualify for this 
program." 11 Interestingly, although the proportion of students identified as 
special education significantly differs across award status and VAM quartiles, it 
does not significantly differ across location. Special education populations range 
from a high of 15 percent of students in IL schools to a low of 7 percent of 
students served by schools in CA. 

Conclusion 

This chapter compares baseline performance, school, and student characteristics by 
award status, VAM performance, and school location for charter schools participating in 
EPIC. The baseline data show charter school award winners and top quartile schools to be 
very similar to poorer performing schools in terms of NCLB performance and other school 
characteristics. The only student characteristics that significantly differ across award status 
and VAM quartile are FRPL eligibility and special education status — schools that performed 
well in Year 1 served higher proportions of both types of students, on average. A number of 
pronounced differences in school and student characteristics were found by location. 



29 Although IL had the highest percent FRPL and was significantly different than other states (p— .01), 
seven schools in IL are missing FRPL data, which suggests that the results should be interpreted cautiously. 
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Chapter IV 



DC Public Schools Baseline 
Statistics 



W e begin this chapter with an overview of Together Everyone Achieves More 
(TEAM) in the District of Columbia, including its relationship to EPIC Year 1 
and Year 2 performance awards, and possible future changes to the program. 
Next, we discuss our plans for evaluating TEAM in DC. Finally, we present our 
findings on DC baseline (Year 1) characteristics, which include information on school 
performance measures, as well as school and student characteristics. We compare these 
characteristics by Year 1 award status to determine whether schools that won awards varied 
systematically at baseline from those that did not. Although school awards in Years 1 and 2 
were not based on a VAM in DC, we also attempt to detect any variation in characteristics 
across VAM quartiles, given that DC plans to switch to a value-added model similar to the 
one currently used in Memphis and charter schools in 2009. 



Overview of TEAM in DC 

Beginning in 2007, all traditional public schools in the District of Columbia Public 
Schools (DCPS) system were eligible for TEAM awards. 1 " In December 2007, Mayor 
Adrian Fenty and Schools Chancellor Michelle Rhee presented the first set of awards to 
three elementary schools. Unlike the Memphis and charter sites, which distributed awards 
based on performance in the VAM, DC linked awards to increases in proficiency rates 
between the 2005/2006 and the 2006/2007 school years. Each of the award winning schools 
experienced more than a 20-percentage point gain in students attaining proficiency in both 
math and English Language Arts. In September 2008, the second round of awards went to 
six DC schools, again based on proficiency rate gains. Three award winners were elementary 
schools, one was a middle school, one was a high school and one served ungraded students. 



30 In DC, the awards are referred to as TEAM awards instead of EPIC awards, which is the term used for 
Memphis and Charter schools. Public charter schools were not eligible for a TEAM award, although some 
competed in the EPIC charter school consortium. 
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These schools increased the percentage of students who were proficient in reading by 
between 20 and 48 percentage points between the 2006/2007 and 2007/2008 school years. 
They showed equally striking improvements in math proficiency. 31 

The distribution and magnitude of the TEAM awards in DC differed from those of the 
EPIC awards in Memphis and charter schools. First, DC distributed awards to non- 
instructional staff, including guidance counselors ($4,000) and support staff ($2,000). 
Second, only Gold-Gain awards were made in DC; no schools received Silver-Gain awards. 
Finally, while the principal and assistant principal awards ($10,000 and $9,000, respectively) 
were similar to those in Memphis, teachers at the award-winning DC schools each received 
$8,000, compared with $1,500 per teacher at Gold-Gain Memphis and charter schools in the 
2007-2008 school year. 

Some aspects of the TEAM program are likely to change in the future. First, DC plans 
to begin using a VAM to evaluate schools that is similar to the one currently used in 
Memphis and the charters. Second, staff at the DC central office have informed MPR that 
they are in the process of closing or restructuring many of their “alternative schools,” 
including schools housed within correctional facilities, family centers, and special education 
schools. Thus, for the purpose of describing baseline schools and evaluating TEAM/EPIC 
in the future, DCPS has requested that we focus only on traditional public schools. ’ 2 



Evaluation of TEAM in DC Schools 

To estimate the impact of incentives on student outcomes in DC, we must compare 
outcomes from eligible and ineligible schools. Because all traditional public schools were 
eligible for awards in DC, there was no relevant comparison group of nonparticipants. The 
only exception was the charter school consortium, which is a poor comparison group. 
Potentially large differences between the charter and non-charter schools at baseline were 
compounded by some DC charter schools’ eligibility for the EPIC charter program and the 
fact that DCPS had a new chancellor (Michelle Rliee), who could affect results for the 
regular public schools, compared to the charters during this period. As a result, we do not 
plan to look for causal links between the TEAM program and student outcomes in DC. 

Although we currently cannot directly estimate the impact of incentives on student 
outcomes in DC, we can, in future reports, offer insight into whether the incentives drive 
teachers to attempt to improve their practices. For incentives to have an impact, educators 
must be aware that they exist, want to improve their own performance because of the 
incentives, and believe that they can do so. Using information from the spring 2008 principal 



31 This baseline report examines only the schools that won awards in the first year because data for the 
second year is not yet available. 

32 This report is based on MPR's understanding as of 10/1/2008 of which schools were "traditional" 
schools in the fall of 2008. We have since been told that the following schools should also be included in our 
analyses: Marshall EC, Reed LC, Takoma EC, Walker-Jones/R.H. Terrell EC, and Winston EC. The schools 
ending in "EC" are "Education Centers" that serve atypical grade configurations (e.g. K-8). The school ending 
in "LC" is a "Learning Center" that is an elementary school embedded in a community center. 
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survey and the planned 2009 teacher survey, we can examine all of these issues and estimate 
the fractions of school staff who are aware of EPIC, who want to improve performance 
because of EPIC awards, and who believe that they can do so. 

Baseline Characteristics 

In the remaining portion of this chapter, we compare baseline characteristics to 
determine if there are significant differences across award status and performance quartiles 
on the VAM. 3 '’ The baseline characteristics fall into three broad categories: (1) school 
performance measures, (2) school characteristics, and (3) student characteristics. 

School performance and school characteristics data were obtained from publicly 
available data. To estimate student characteristics, we created a data set of student -level data 
for each school for the students who were included in the VAM data set. 

The following characteristics are measured: 

• School Performance Characteristics, the proportion of schools in each 
category making Adequate Yearly Progress (AYP), percentage of schools 
identified as in need of improvement (SINI), and the average percentage of 
students scoring proficient on the District of Columbia Comprehensive 
Assessment System (DC-CAS) math and English Language Arts exams. These 
data are for the 2006-2007 school year. 

• School Characteristics, the total number of students, the student/ teacher 
ratio, and the total number of teachers. These data are from the Common Core 
of Data for the 2005-2006 school year which was the most recent year of data 
available at the time of this report. 

• Student Characteristics, the percentage of students who are black, Hispanic, 
eligible for free or reduced lunch (FRPL), limited English proficient (LEP), and 
identified as being in special education. These data are for students used to 
calculate the VAM scores and are for the 2006-2007 school year. 

Since sample sizes are small, even large variations across groups are unlikely to generate 
much statistical significance because of the likelihood that these differences occurred by 
chance. This is particularly true for comparisons between award and non-award winners. 



33 VAM performance was measured for the 2006-2007 school year using methods similar to that used in 
Memphis. See Booker and Isenberg (2008) for details. 
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Baseline Characteristics by Award Status 

Table IV. 1 shows the baseline characteristics of schools by award status. Only three 
schools won awards in 2007, and these are compared to 129 non-award winning schools. 



Table IV.1. Baseline Statistics by Award Status, DC Public Schools 



Category 


Award Recipients 


Non-Award 

Schools 


School Performance (%) 


AYP pass 


100.0 


22.8“* 


SINI 


0.0 


78.7*“ 


Student proficiency 
ELA proficiency 


56.5 


38.2 


Math proficiency 


45.1 


30.9 


School Characteristics 


Enrollment 


287.3 


382.1 


Student-teacher ratio 


17.1 


13.6 


Staff size 


17.0 


31.3 


Student Characteristics (%) 
Ethnicity 
Black 


92.2 


83.6 


Hispanic 


6.6 


10.2 


FRPL 


82.9 


65.8 


LEP 


7.8 


6.4 


Special education 


16.6 


15.5 


Number of Schools 


3 


129 



‘Significantly different from the award recipients at the 10% level 
‘‘Significantly different from the award recipients at the 5% level. 
‘“Significantly different from the award recipients at the 1% level. 



Note: School performance measures and school characteristics were obtained from publicly 

available data (Common Core of Data for 2005-2006 school year). Student 
characteristics were averaged across all students included in the VAM (2006-2007 school 
year). 

• Performance Characteristics. Significant differences did exist with respect to 
performance measures. Award winning schools were much more likely to make 
AYP in this period year (100 percent) than non award winners (23 percent) and 
much less likely to be classified as in need of improvement for failing to make 
AYP in consecutive years (0 percent of award schools compared with 79 
percent of non award schools). The differences along NCLB performance 
measures may be explained in part by the structure of the DC awards. Like the 
DC awards metric, the “Safe-Harbor” provision of NCLB enables schools to 
make AYP based on the increase in the percentage of students scoring 
proficient on state exams. As a result, schools that make large gains in percent 
proficient are also likely to make AYP. 
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• School and Student Characteristics. In general, schools that won awards and 
schools that did not win awards are similar. On average, between 6 and 8 
percent of students are LEP and between 6 and 11 percent were Hispanic. Just 
over 15 percent were identified for special education. Although a higher 
proportion of students in the schools that won awards were black (92 percent 
compared to 84 percent) and eligible for FRPL (83 percent compared to 66 
percent), there was no significant variation in these characteristics across 
categories and it is possible that this variation occurred by chance. Similarly, 
although award winning schools tended to be smaller (287 average students 
compared to 382 in non-award schools) there was no significant relationship 
between award status and school size. 

Of 12 comparisons by award status, only 2 were significantly different. These 
differences were limited to performance on NCLB-related measures. School and student 
characteristics did not vary significantly by award status. 

Baseline Characteristics by VAM Quartile Scores 

Table IV.2 shows the baseline characteristics by VAM quartile. Although awards in DC 
schools were not based on the VAM used in the other sites, the comparison is relevant, since 
DC plans to switch to the VAM in 2009. Better performing schools in the VAM would not 
necessarily win awards in 2007 under the proficiency-based DC award model. Schools first 
were separated by school level (elementary, middle, high) and then placed in quartiles based 
on performance in MPR’s VAM. For each school level, we combined the two middle 
quartiles and then merged the different school levels to produce a total of 65 schools in the 
middle quartiles, compared with 30 in each of the top and bottom quartiles. All tests of 
statistical significance were between the top or bottom quartile and the middle two quartiles. 

• Performance Characteristics. Schools in the top quartile were significantly 
different from those in the middle quartiles in three of the four characteristics 
measured. The top quartile of all schools was much less likely than the middle 
quartiles to be designated as in need of improvement (67 percent of top quartile 
schools needed improvement, compared to 82 percent of schools in the middle 
quartile). Similarly, the average percentage of students scoring proficient on 
English Language Arts and math tests was significantly higher for schools in the 
top quartile of the VAM. For English language, average percent proficient 
ranged from 48 percent of students in the top quartile to only 29 percent in the 
bottom quartile. For math, the top quartile averaged 42 percent proficient, while 
the bottom quartile averaged 21 percent. Across all quartiles, only a small 
proportion of schools made AYP, even in the top quartile. It is interesting to 
note that even schools that increased their value-added scores by the greatest 
margins still had difficulty meeting the requirements of NCLB. 
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Table IV.2. Baseline Statistics by VAM Quartile Scores, DC Public Schools 



Category 


Top Quartile 


Middle Quartile 


Bottom Quartile 


School Performance (%) 


AYP pass 


33.3 


18.5 


20.0 


SINI 


66.7* 


81.5 


86.7 


Student proficiency 
ELA proficient 


48.0“ 


36.6 


29.1* 


Math proficient 


42.2“* 


28.7 


21.3* 


School Characteristics 


Enrollment 


397.5 


379.3 


393.9 


Student-teacher ratio 


14.0 


13.1 


14.5 


Staff size 


32.7 


31.8 


30.3 


Student Characteristics (%) 
Ethnicity 
Black 


69.1“* 


88.7 


93.9 


Hispanic 


17.3“ 


8.6 


5.8* 


FRPL 


61.6 


66.7 


75.1 


LEP 


10.2* 


5.7 


3.6 


Special education 


14.5 


15.9 


16.8 


Number of Schools 


30 


65 


30 



‘Significantly different from middle quartile schools at the 10% level. 

“Significantly different from middle quartile schools at the 5% level. 

“‘Significantly different from middle quartile schools at the 1% level. 

Note: School performance measures and school characteristics were obtained from publicly 

available data (Common Core of Data for 2005-2006 school year). Student 
characteristics were averaged across all students included in the VAM (2006-2007 school 
year). VAM quartiles were based on 2006-2007 school year performance. 

• School Characteristics. Across all school levels, there were no significant 
differences between quartiles in terms of school characteristics. Average school 
size ranged between 379 and 397 for all schools — as expected, elementary 
schools tended to be smaller and high schools tended to be larger. 
Student/ teacher ratios averaged around 14. 

• Student Characteristics. Student characteristics, on the other hand, varied 
significantly by quartile. Out of 10 comparisons by VAM quartile, 4 were 
significant. Schools in the top quartile served a significantly lower proportion of 
students who were black; schools in the top quartile averaged 69 percent black 
students, compared to 89 percent black in the middle quartiles and 94 percent 
black in the bottom quartile. Top quartile schools also served a higher 
percentage of students who were Hispanic and LEP; on average, schools in the 
top quartile served a student body that was 17 percent Hispanic and 10 percent 
LEP, compared to 9 percent Hispanic and 6 percent LEP for schools in the 
middle quartiles. One other difference distinguished the bottom quartile. These 
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schools served a higher proportion of students eligible for FRPL (75 percent 
compared to 67 percent in the middle quartiles). 

Appendix Tables A. 10 to A. 12 show similar VAM comparisons by school level 
(elementary, middle and high). In general, these tables show very few significant differences, 
with the exception of elementary schools. Of the 24 comparisons considered for each level, 
one is significant for high schools (the percentage of students identified for special education 
is significantly larger for schools in the bottom quartile than those in the middle quartiles) 
and only 2 are significant for middle schools. For elementary schools, six significant 
relationships exist between VAM quartiles and baseline characteristics, but all of these 
differences also exist when school levels are combined. In many cases, the differences are 
greater in size for individual school levels, but become significant only in the combined 
table. With one exception, differences between quartiles are limited to school performance 
and student characteristics. 

Conclusion 

The baseline data show DC award winning schools to be very similar to non-award 
winning schools in terms of student and school characteristics. Differences in school 
performance might have resulted from the similarity between the Safe-Flarbor criteria for 
making AYP and the stmcture of the DC awards, which are based on increases in school 
proficiency rates. 

Differences in baseline characteristics across VAM quartiles were more prevalent than 
in comparisons across award status. Since the VAM measures average improvement in 
student test scores instead of the change in percent proficient at a school level, the link 
between performance on the VAM and performance on NCLB is not as direct; schools 
could make dramatic improvements in student value-added scores but might still fail to make 
AYP, or students might not meet proficiency cutoffs on math and English Language Arts 
exams. It is likely, however, that schools making large value-added gains also would increase 
the proportion of students meeting proficiency standards. 

School characteristics were not significantly different across value-added quartiles in 
DC, but student characteristics did vary significantly for different quartiles, particularly 
among elementary and high schools. This is particularly interesting, since similar differences 
did not exist between schools that won awards in the DC model and those that did not. In 
fact, the small number of schools that won awards in DC actually had a higher proportion of 
students who were black and eligible for FRPL, in direct contrast to the schools in the top 
quartile of the VAM, although these differences were not significant by award status. 
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Table A.1. Student Sample vs. Overall School Characteristics MCS That Include Only 
Tested Grades 





Tested Students (%) 


All Students (%) 


Ethnicity 






Black 


95.4 


94.5* 


Hispanic 


2.9 


3.0 


FRPL 


85.1 


84.8 


Male 


50.7 


52.0** 


Number of Schools 


19 





Note: Includes schools whose grades served fall between 3 and 8. 

The data for tested students are for the students in our VAMs for the 2006-2007 school 
year. The data for all students are from the Common Core of Data for 2005-2006 school 
year. 
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Table A.2. Student Sample vs. Overall School Characteristics Charter Schools That 
Include Only Tested Grades 



Characteristic 


Tested Students 

(%) 


All 

Students 

(%) 


Number of Schools with 
Nonmissing Values for Both 
VARS 


Ethnicity 








Black 


52.1 


52.2 


7 


Hispanic 


31.9 


26.8 


7 


FRPL 


65.2 


67.7 


7 


LEP 


12.2 


16.2 


5 


Special Education 


13.8 


9.5* 


5 


Male 


49.6 


48.2 


4 



‘Significantly different from the tested students at the 10% level. 

“Significantly different from the tested students at the 5% level. 

“‘Significantly different from the tested students at the 1% level. 

Note: Includes schools whose grades served fall between 3 and 8. 

The data for tested students are for the students in our VAMs for the 2006-2007 school 
year. The data for all students are from publicly available data for the 2006-2007 school 
year when possible and from the 2005-2006 school year otherwise. 
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Table A.3. Student Sample vs. Overall School Characteristics DC Schools That Include 
Only Tested Grades 



Characteristic 


Tested Students (%) 


All Students (%) 


Ethnicity 

Black 


86.7 


86.0 


Hispanic 


10.2 


10.3 


FRPL 


69.1 


70.3 


LEP 


5.3 


5.7 


Special Education 


18.2 


19.1 


Male 


48.3 


49.8 


Number of Schools 


12 





‘Significantly different from the tested students at the 10% level. 

“Significantly different from the tested students at the 5% level. 

“‘Significantly different from the tested students at the 1% level. 

Note: Includes schools whose grades served fall between 3 and 8. 

The data for tested students are for the students in our VAMs for the 2006-2007 school 
year. The data for all students are from the Common Core of Data for 2005-2006 school 
year. 
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Table A.4. Baseline Statistics for MEMPHIS Elementary Schools By VAM Quartile 
Scores 



Category 


Top Quartile 


Middle Quartile 


Bottom Quartile 


School Performance 








AYP pass (%) 


100.0 


95.9 


72.0“* 


SINI (%) 


4.2 


6.1 


8.0 


Student proficiency (%) 








ELA proficient 


87.0 


84.4 


79.2*“ 


Math proficient 


87.5“ 


82.9 


75.9*“ 


Promotion rate 


95.7 


95.7 


95.2 


School Characteristics 








Enrollment 


500.7 


564.9 


529.3 


Student-teacher ratio 


16.5 


16.0 


16.4 


Staff size 


30.1* 


34.9 


31.8 


Teacher ethnicity (%) 








White 


34.1 


33.4 


32.2 


Black 


65.1 


64.2 


66.3 


Hispanic 


0.1 


0.4 


0.7 


Master’s or Ph.D. (%) 


61.6 


60.3 


59.1 


Student Characteristics (%) 








Ethnicity 








Black 


92.7 


88.4 


92.7 


Hispanic 


4.1 


54.4 


3.8 


FRPL 


87.3 


86.6 


88.9 


LEP 


3.4 


4.3 


3.2 


Special education 


11.6 


11.2 


11.0 


Number of Schools 


25 


49 


25 



‘Significantly different from the tested students at the 10% level. 

‘‘Significantly different from the tested students at the 5% level. 

‘“Significantly different from the tested students at the 1% level. 

Notes: Only Memphis schools eligible for an EPIC award in 2008-2009 are included in these 
statistics. 

School performance measures and school characteristics were obtained from publicly 
available data (Common Core of Data for 2005-2006 school year). Student 
characteristics were averaged across all students included in the VAM (2006-2007 school 
year). VAM quartiles were based on 2006-2007 school year performance. 
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Table A.5. Baseline Statistics for MEMPHIS Middle Schools by VAM Quartile Scores 



Category 


Top Quartile 


Middle Quartile 


Bottom Quartile 


School Performance 








AYP pass (%) 


83.3 


63.6 


20.0 


SINI (%) 


20.0 


27.3 


40.0 


Student proficiency (%) 








ELA proficient 


82.4 


83.3 


80.0 


Math proficient 


79.2 


80.4 


77.8 


Promotion rate 


96.9“ 


90.1 


89.7 


School Characteristics 








Enrollment 


737.3 


867.0 


791.0 


Student-teacher ratio 


16.5 


18.5 


16.4 


Staff size 


41.2 


46.3 


46.5 


Teacher ethnicity (%) 








White 


22.3 


24.5 


24.4 


Black 


75.1 


73.8 


75.5 


Hispanic 


0.2 


1.0 


0.1 


Master’s or Ph.D. (%) 


68.2 


55.2 


65.2 


Student Characteristics (%) 








Ethnicity 








Black 


95.3 


93.8 


95.9 


Hispanic 


2.3 


3.1 


2.3 


FRPL 


84.6 


81.1 


89.7 


LEP 


1.4 


81.1 


89.7 


Special education 


10.7 


10.6 


12.2 


Number of Schools 


6 


11 


5 



‘Significantly different from the tested students at the 10% level. 

“Significantly different from the tested students at the 5% level. 

“‘Significantly different from the tested students at the 1% level. 

Notes: Only Memphis schools eligible for an EPIC award in 2008-2009 are included in these 
statistics. 

School performance measures and school characteristics were obtained from publicly 
available data (Common Core of Data for 2005-2006 school year). Student 
characteristics were averaged across all students included in the VAM (2006-2007 school 
year). VAM quartiles were based on 2006-2007 school year performance. 
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Table A.6. Baseline Statistics for MEMPHIS High Schools by VAM Quartile Scores 



Category 


Top Quartile 


Middle Quartile 


Bottom Quartile 


School Performance 








AYP pass (%) 


50.0 


33.3 


50.0 


SINI (%) 


25.0 


55.6 


25.0 


Student proficiency (%) 








ELA proficient 


87.5 


86.5 


87.5 


Math proficient 


79.8 


73.5 


79.8“ 


Promotion rate 


82.1“ 


77.4 


82.1 


School Characteristics 








Enrollment 


1,356.8 


1,120.7 


1,356.8 


Student-teacher ratio 


20.4 


2.8 


20.4 


Staff size 


66.3 


53.1 


66.3 


Teacher ethnicity (%) 








White 


28.1 


33.4 


28.1 


Black 


68.7 


64.8 


68.7 


Hispanic 


1.3 


0.3 


1.3 


Master’s or Ph.D. (%) 


59.9 


62.3 


59.9 


Student Characteristics (%) 








Ethnicity 








Black 


96.5 


95.9 


96.5* 


Hispanic 


2.3 


1.9 


2.3 


FRPL 


75.1 


80.5 


75.1 


LEP 


1.9 


1.1 


1.9 


Special education 


7.6 


9.1 


7.6 


Number of Schools 


4 


9 


4 



‘Significantly different from the tested students at the 10% level. 

‘‘Significantly different from the tested students at the 5% level. 

‘“Significantly different from the tested students at the 1% level. 

Notes: Only Memphis schools eligible for an EPIC award in 2008-2009 are included in these 
statistics. 

School performance measures and school characteristics were obtained from publicly 
available data (Common Core of Data for 2005-2006 school year). Student 
characteristics were averaged across all students included in the VAM (2006-2007 school 
year). VAM quartiles were based on 2006-2007 school year performance. 
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Table A.7. Baseline Statistics for Charter Elementary Schools by VAM Quartiles 



Category 


Top Quartile 


Middle Quartile 


Bottom Quartile 


School Performance (%) 


AYP pass 


71.4 


56.3 


33.3 


SINI 


28.6 


25.0 


50.0 


School Characteristics 


Enrollment 


224.1 


245.9 


471.4“* 


Student-teacher ratio 


17.8 


15.6 


21.1“ 


Staff size 


15.0 


17.2 


27.1“ 


Years operating 
2 years or fewer 


0.0 


0.0 


0.0 


3-5 years 


50.0 


47.1 


42.9 


6 years or more 


50.0 


52.9 


57.1 


Student Characteristics (%) 
Ethnicity 
Black 


49.8 


55.1 


84.5“ 


Hispanic 


42.5“ 


16.7 


10.6 


FRPL 


76.5 


70.0 


74.0 


LEP 


12.8 


6.6 


4.3 


Special education 


22.4* 


11.7 


9.8 


Number of Schools 


8 


17 


8 



‘Significantly different from the tested students at the 10% level 
‘‘Significantly different from the tested students at the 5% level. 
‘“Significantly different from the tested students at the 1% level. 



Note: School performance measures and school characteristics were obtained from publicly 

available data — 2006-2007 when possible and 2005-2006 otherwise. Student 
characteristics were averaged across all students included in the VAM (2006-2007 school 
year). VAM quartiles were based on 2006-2007 school year performance. 
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Table A.8. Baseline Statistics for Charter Middle Schools by VAM Quartiles 



Category 


Top Quartile 


Middle Quartile 


Bottom Quartile 


School Performance (%) 


AYP pass 


72.7 


69.6 


30.0* 


SINI 


18.2 


39.1 


30.0 


School Characteristics 


Enrollment 


329.2“ 


539.4 


550.2 


Student-teacher ratio 


16.1 


18.3 


17.6 


Staff size 


24.1* 


32.5 


31.3 


Years operating 
2 years or fewer 


0.0 


4.2 


9.1 


3-5 years 


36.4 


25.0 


18.2 


6 years or more 


63.6 


70.8 


72.7 


Student Characteristics (%) 
Ethnicity 
Black 


46.4 


52.3 


35.6 


Hispanic 


25.7 


27.2 


17.6 


FRPL 


72.0 


59.8 


45.8 


LEP 


13.0 


7.1 


5.2 


Special education 


11.6 


9.3 


7.2 


Number of Schools 


11 


24 


11 



‘Significantly different from the tested students at the 10% level 
‘‘Significantly different from the tested students at the 5% level. 
‘“Significantly different from the tested students at the 1% level. 



Note: School performance measures and school characteristics were obtained from publicly 

available data — 2006-2007 when possible and 2005-2006 otherwise. Student 
characteristics were averaged across all students included in the VAM (2006-2007 school 
year). VAM quartiles were based on 2006-2007 school year performance. 
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Table A.9. Baseline Statistics for Charter High Schools by VAM Quartiles 



Category 


Top Quartile 


Middle Quartile 


Bottom Quartile 


School Performance (%) 


AYP pass 


50.0 


30.0 


50.0 


SINI 


100.0 


40.0 


50.0 


School Characteristics 


Enrollment 


445.5 


476.8 


484.0 


Student-teacher ratio 


16.8 


15.0 


14.0 


Staff size 


37.0 


30.7 


40.0 


Years operating 
2 years or fewer 


25.0 


10.0 


0.0 


3-5 years 


0.0 


40.0 


75.0 


6 years or more 


75.0 


50.0 


25.0 


Student Characteristics (%) 
Ethnicity 
Black 


66.9 


30.0 


34.2 


Hispanic 


28.3 


31.1 


48.1 


FRPL 


79.6 


62.8 


69.5 


LEP 


0.4 


5.0 


1.0 


Special education 


7.0 


5.5 


14.9“ 


Number of Schools 


4 


10 


4 



‘Significantly different from the tested students at the 10% level 
‘‘Significantly different from the tested students at the 5% level. 
‘“Significantly different from the tested students at the 1% level. 



Note: School performance measures and school characteristics were obtained from publicly 

available data — 2006-2007 when possible and 2005-2006 otherwise. Student 
characteristics were averaged across all students included in the VAM (2006-2007 school 
year). VAM quartiles were based on 2006-2007 school year performance. 
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Table A.10. Baseline Statistics for DC Elementary Schools By VAM Quartile Scores 



Category 


Top Quartile 


Middle Quartile 


Bottom Quartile 


School Performance (%) 


AYP pass 


36.4 


21.7 


27.3 


SINI 


63.6* 


82.6 


81.8 


Student proficiency 
ELA proficient 


50.7“* 


37.7 


34.2 


Math proficient 


43.1*“ 


27.5 


24.3 


School Characteristics 


Enrollment 


329.8 


307.4 


332.0 


Student-teacher ratio 


14.7 


13.8 


14.6 


Staff size 


24.6 


23.8 


24.7 


Student Characteristics (%) 
Ethnicity 
Black 


65.2*** 


89.5 


93.0 


Hispanic 


18.4“ 


8.3 


6.6 


FRPL 


61.9 


69.5 


77.1 


LEP 


11.8* 


6.0 


4.5 


Special education 


15.1 


16.0 


15.4 


Number of Schools 


22 


46 


22 



‘Significantly different from the tested students at the 10% level 
“Significantly different from the tested students at the 5% level. 
“‘Significantly different from the tested students at the 1% level. 



Note: School performance measures and school characteristics were obtained from publicly 

available data (Common Core of Data for 2005-2006 school year). Student 
characteristics were averaged across all students included in the VAM (2006-2007 school 
year). VAM quartiles were based on 2006-2007 school year performance. 
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Table A.11. Baseline Statistics for DC Middle Schools By VAM Quartile Scores 



Category 


Top Quartile 


Middle Quartile 


Bottom Quartile 


School Performance (%) 


AYP pass 


25.0* 


0.0 


0.0 


SINI 


75.0 


90.0 


100.0 


Student proficiency 
ELA proficient 


36.8 


25.8 


15.4 


Math proficient 


39.7 


23.4 


13.2 


School Characteristics 


Enrollment 


550.8“ 


362.0 


498.3 


Student-teacher ratio 


13.8 


12.4 


13.5 


Staff size 


40.3 


31.0 


37.0 


Student Characteristics (%) 
Ethnicity 
Black 


79.1 


85.4 


99.3 


Hispanic 


10.6 


11.6 


0.7 


FRPL 


67.0 


68.0 


75.0 


LEP 


4.1 


6.4 


0.9 


Special education 


16.4 


19.9 


19.6 


Number of Schools 


4 


10 


4 



‘Significantly different from the tested students at the 10% level 
“Significantly different from the tested students at the 5% level. 
“‘Significantly different from the tested students at the 1% level. 



Note: School performance measures and school characteristics were obtained from publicly 

available data (Common Core of Data for 2005-2006 school year). Student 
characteristics were averaged across all students included in the VAM (2006-2007 school 
year). VAM quartiles were based on 2006-2007 school year performance. 
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Table A.12. Baseline Statistics for DC High Schools By VAM Quartile Scores 



Category 


Top Quartile 


Middle Quartile 


Bottom Quartile 


School Performance (%) 








AYP pass 


25.0* 


22.2 


0.0 


SINI 


75.0 


66.7 


100.0 


Student proficiency 








ELA proficient 


43.2 


42.2 


9.8 


Math proficient 


38.7 


40.0 


9.9 


School Characteristics 








Enrollment 


616.5“ 


765.8 


630.3 


Student-teacher ratio 


9.8 


10.4 


14.9 


Staff size 


70.0 


74.0 


54.3 


Student Characteristics (%) 








Ethnicity 








Black 


81.1 


88.1 


93.4 


Hispanic 


17.6 


6.4 


6.6 


FRPL 


53.7 


49.2 


61.6 


LEP 


7.7 


2.7 


1.0 


Special education 


9.4 


10.5 


22.8“ 


Number of Schools 


4 


9 


4 



‘Significantly different from the tested students at the 10% level 
“Significantly different from the tested students at the 5% level. 
“‘Significantly different from the tested students at the 1% level. 



Note: School performance measures and school characteristics were obtained from publicly 

available data (Common Core of Data for 2005-2006 school year). Student 
characteristics were averaged across all students included in the VAM (2006-2007 school 
year). VAM quartiles were based on 2006-2007 school year performance. 
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